Map of Pictures
Squeeze every picture into just two numbers and you can plot it on a map. Click anywhere on the map and the network draws what lives there.
I am learning a little map. Similar pictures should end up close together. When the dots settle, click or drag on the map and I will draw that spot!
The map
Faint tiles show what the network draws at each spot. Dots are the pictures it learned from. Arrow keys move the spot too.
Challenges
Tidy map: train until at least 65% of dots have neighbours of the same shape.
Explorer: click spots on the map that the network says look like 4 different shapes.
Blend: press New A and New B until the two kinds differ, then slide the mix to the middle.
Empty land: click a spot far from every dot and see what the network invents there.
What just happened?
- Each picture became just two numbers, so it could be a dot on a flat map. Nobody told the network which shapes exist, but similar shapes still landed near each other.
- The second half of the network turns any spot on the map back into a picture, even a spot where no real picture was.
- Sliding along the map gives a smooth change of shape. Plain cross-fading just makes a ghostly double image.
- Far from the dots the drawings get odd, because the network never practised there. That is exactly the problem the next lesson fixes.
For grown-ups: what's really going on
The 2-unit bottleneck of this autoencoder (64 → 32 → 2 → 32 → 64) is a latent space: a learned low-dimensional coordinate system for the data. The encoder maps a picture to a point; the decoder maps any point back to a picture.
- It is trained only to reconstruct. Clustering by shape is a side-effect: the encoder must give different pictures different codes, and similar pictures need similar codes to be rebuilt similarly.
- "Tidy" is a leave-one-out k-nearest-neighbours check (k = 5) on the 360 training points. With 6 overlapping classes and only 2 dimensions, perfect is not expected.
- Interpolation: decoding points on a straight line between two codes gives smooth in-betweens. Pixel cross-fading is linear in pixel space and just blends the two images.
- A plain autoencoder places no rule on empty areas of the map, so decoded pictures there can be junk. Variational autoencoders add such a rule.
Further reading: Bengio, Courville and Vincent, Representation Learning: A Review and New Perspectives (2013).