Teach the Camera
My eyes are a picture brain called MobileNet. Press Download to fetch it (about 15 MB), then teach me!
Robo's eyes aren't downloaded yet. It's the MobileNet vision model: about 15 MB (1.5 MB engine + 14 MB of learned numbers). Nothing downloads until you press the button.
Teach me first!
Add a few examples to at least two groups.
What's happening inside
1 · What Robo sees
2 · Turned into 1,280 numbers
3 · 5 closest examples
4 · They vote
Teach me first!
Robo's idea map
Every example becomes a dot. Things that look alike to Robo land close together. The star is what Robo sees right now.
How Robo decides
1Robo already learned to see shapes and colors from a million photos. It turns any picture into a list of 1,280 numbers — its "description".
2When you record, Robo saves the description of each example. That's the learning: it takes less than a second!
3To guess, Robo finds the few examples most like what it sees now (up to 5), and they vote. The closest ones get the loudest votes!
Examples
0
Guess speed
– ms
How it works
Make groups
Name the things you want Robo to tell apart, like "thumbs up" and "just me".
Show examples
Hold the button and wiggle around. More examples, from different angles, make Robo smarter.
Closest friends vote
Robo compares what it sees to all your examples. The most similar ones decide the answer.
Try to trick it
Change the light, move far away, or show something new. When does Robo get confused? Teach it more!
What just happened?
- You made groups and showed Robo examples of each one. That's called training data.
- Robo didn't learn to see from zero: it borrowed eyes that already studied a million photos, and turned each example into a long list of numbers.
- To guess, it found the examples most like the new picture and let them vote.
- The checkpoints showed that more, varied examples make Robo surer — and that it can still be fooled by things it never saw.
For grown-ups: what's really going on
- This is transfer learning, the same trick as Google's Teachable Machine. A MobileNet v2 CNN pretrained on ImageNet (1.28M images, 1000 classes) runs in your browser via TensorFlow.js.
- We drop its final classification layer and use the penultimate activations (
mobilenet.infer(img, true)) as an embedding: a 1280-dimensional feature vector that captures visual concepts. - The classifier is a hand-written k-nearest neighbours using cosine similarity on L2-normalised embeddings, with k = min(5, smallest class size). Each neighbour votes with weight
exp(20·(sᵢ − s_max)), a softmax over similarities; confidence is the normalised vote share. The "self-quiz" is leave-one-out accuracy on (up to 80 of) your own examples. No gradient descent is needed, which is why "training" is instant. - The idea map is a live PCA (top two principal components, found by power iteration with deflation) of all stored embeddings. Clusters that separate in 2D are usually easy for KNN; overlapping ones are where it gets confused.
- Failure modes worth exploring: background/lighting shortcuts (the model may learn "dark room" rather than "hand"), class imbalance, and too few varied examples. All images stay on your device.