Make New Pictures
Pick random spots on the map and the network draws brand-new pictures. A tidy-up slider decides whether the map is full of holes or a blurry mush.
I will draw pictures from random spots. If the map has holes I draw junk. Slide the tidy-up weight and watch the junk and the blur change!
The map
Faint tiles are what the network draws at each spot. Dots are real pictures. The dashed circle is where random spots usually land. White rings are your new pictures.
0 = no rule about the map. Higher = it must keep the dots packed in the circle, at the cost of detail.
Challenges
No rules: set the weight to 0 and let it train until Spread is over 20. The dots sprawl far outside the circle, leaving holes.
Sweet spot: find a weight where junk is 30% or less and the pictures stay sharp (0.7 or more).
Too much tidying: set the weight to 3 and see sharpness drop to 0.65 or less.
Experimenter: train with 4 different weights.
What just happened?
- This network is a variational autoencoder. Instead of one exact spot, it describes each picture as a fuzzy blob (a centre plus some random wobble), then rebuilds the picture from a random point inside that blob.
- The tidy-up weight pulls all blobs towards the middle, so random spots land on something real. It is a trade: too little leaves holes and junk, too much makes every picture a grey, blurry average.
- New pictures come from picking a random spot in the circle and decoding it. Nobody drew those pictures.
- The junk and sharp numbers are measured by comparing each new picture with every real one, so they are not faked.
For grown-ups: what's really going on
A variational autoencoder (VAE) has an encoder that outputs a mean μ and log-variance log σ² for a 2-D Gaussian, and a decoder (2 → 32 → 64). Training minimises ‖x − x̂‖² + β · KL.
- Reparameterisation trick:
z = μ + σ · εwithε ~ N(0, 1), so gradients flow through the sampling step into the encoder. This is implemented by hand here. - KL divergence to the prior
N(0, I):−½ Σ (1 + log σ² − μ² − σ²). It is the regulariser. With β = 0 it is a plain autoencoder: the codes spread far outside the prior, so sampling fromN(0, I)often lands in untrained gaps (junk). - Large β causes posterior collapse in the limit: the encoder ignores the input, and the decoder outputs the average picture.
- This is a β-VAE view: the weight is on the KL term relative to the squared-error reconstruction term (a Gaussian likelihood with a fixed variance).
- "Junk" = nearest training picture has per-pixel mean squared difference above 0.05. "Sharp" = mean of
|pixel − 0.5| × 2: 1 for pure black and white, 0 for mid grey.
Further reading: Kingma and Welling, Auto-Encoding Variational Bayes (2014); Higgins et al., β-VAE (2017).