Computer Vision

What Is It?

I studied over a million photos of 1,000 different things. First press Download to fetch my picture brain (about 15 MB), then show me a picture!

What's happening inside

Robo's brain is a stack of about 50 layers. Each layer looks at what the one before found, and finds something a bit bigger.

Start

Pixels

224 × 224 squares × 3 colors = 150,528 numbers go in.

Early layers

Edges & lines

First, find where colors change (like the magic glasses in Robot Eyes).

Middle layers

Important parts

Edges build into shapes and parts: ears, wheels, peel. Press Show me where you look to see which parts mattered.

Last layer

1,000 scores

One score for every thing Robo knows. The tallest wins: …

How it works

Study lots

Grown-ups showed this brain over a million photos, each with a label like "banana" or "goldfish".

Layer by layer

Edges become shapes, shapes become parts, parts become whole things.

Score everything

At the end it gives a score to all 1,000 things it knows. The biggest score is its guess.

It only knows what it saw

Never saw a drawing or a unicorn? Then it'll guess something silly. Try it!

What just happened?

For grown-ups: what's really going on
  • The model is MobileNet v2 (α = 1.0, 224×224 input), a convolutional neural network trained on ImageNet-1k, running in-browser with TensorFlow.js (WebGL backend when available). It uses depthwise-separable convolutions and inverted residual blocks to stay small (~3.5M parameters).
  • We centre-crop your image to a square, resize to 224×224, scale pixels to [−1, 1], and take the softmax over the 1000 class logits. The bar chart in "Last layer" is that full probability vector, in class-index order.
  • "Where does Robo look?" is occlusion sensitivity (Zeiler & Fergus, 2014): slide a gray patch with 50% overlap, re-run the network on each occluded copy (batched), and record the drop in the top class's probability. It's model-agnostic and honest, but coarse; gradient methods like Grad-CAM give finer maps.
  • The "Edges" panel is a Sobel filter on your image, shown as an analogy: real first-layer MobileNet filters learn similar oriented-edge and colour-blob detectors.
  • Known limits: ImageNet is photo-centric with many dog breeds (~120 of 1000 classes) and no "person" class, so drawings and people get odd guesses. Dataset bias shows up directly in what it can and can't name.
  • Further reading: CNN Explainer (Polo Club), Feature Visualization (Distill). Sample photos: Wikimedia Commons.