Diffusion · Main lesson

Watch a Diffusion Model Learn

A brand-new baby brain only knows how to make fog. Press play and watch it practise cleaning up noisy dots, getting better every round, until it can make your shape out of nothing.

Pick a shape, then press Start learning!

Making dots step 0 brain

Checkpoints

Every so often the brain tries making dots. Tap a card to replay it.
Press Start learning to make the first checkpoint.

What's happening inside

1Tap any dot in the big picture to follow it.
2The brain looks at where the dot is and how fuzzy things are.
3It guesses which way the noise pushed the dot.
4So the dot takes a small step the other way.

How it works

Make a mess on purpose

Take dots from the real shape and shake each one with a random push. Remember the push!

Guess the push

Show the brain the shaken dot and how hard we shook. It guesses which way the push went.

Fix the brain a little

Compare the guess with the real push. Nudge the brain's numbers so next time the guess is a bit closer.

Make something new

Start from pure fog. Ask the brain "which way was the push?", step the other way, and repeat 100 times.

What just happened?

For grown-ups: what's really going on
  • Real training, in your browser. A plain-JavaScript MLP (no libraries) is trained live with backprop and Adam (batch 128, gradient clipping at 1).
  • Objective. The DDPM "simple" loss from Ho et al. 2020: sample x₀ from the dataset, t ~ U{0..99}, ε ~ 𝒩(0, I), form xₜ = √ᾱₜ x₀ + √(1−ᾱₜ) ε, and minimise ‖ε − ε̂θ(xₜ, t)‖². Linear β schedule 1e-4 → 0.08, T = 100.
  • Network. Input is (x, y), Fourier features sin/cos(k·x) for k ∈ {1, 2, 4, 8}, and a sinusoidal timestep embedding (34 numbers). Then 3 ReLU hidden layers of 32 / 64 / 128 units and 2 outputs (ε̂). The loss never reaches 0: even a perfect model can't know the exact noise, only its conditional mean, so the floor is well above 0, especially at high t.
  • Sampling. Ancestral DDPM: xₜ₋₁ = (xₜ − βₜ/√(1−ᾱₜ) · ε̂) / √αₜ + σₜ z, with σₜ² = β̃ₜ. With fewer cleaning steps we use strided DDPM steps (a respaced schedule), as in Nichol & Dhariwal 2021. The arrows show −ε̂ (the score direction, since ∇log pₜ = −ε̂/√(1−ᾱₜ)).
  • Checkpoints. At steps 0, 25, 75, 150 … the weights are snapshotted and 500 samples are drawn. The bar is the overlap between the sample histogram and the data histogram, divided by what genuine data samples score.
  • Inspired by tiny-diffusion. See the Diffusion Explainer (Polo Club) for the same process inside Stable Diffusion, where ε̂ comes from a U-Net working on image latents and is guided by text.