Watch a Diffusion Model Learn
A brand-new baby brain only knows how to make fog. Press play and watch it practise cleaning up noisy dots, getting better every round, until it can make your shape out of nothing.
Pick a shape, then press Start learning!
Making dots
step 0 brain
Checkpoints
Every so often the brain tries making dots. Tap a card to replay it.Press Start learning to make the first checkpoint.
What's happening inside
1Tap any dot in the big picture to follow it.
2The brain looks at where the dot is and how fuzzy things are.
3It guesses which way the noise pushed the dot.
4So the dot takes a small step the other way.
How it works
Make a mess on purpose
Take dots from the real shape and shake each one with a random push. Remember the push!
Guess the push
Show the brain the shaken dot and how hard we shook. It guesses which way the push went.
Fix the brain a little
Compare the guess with the real push. Nudge the brain's numbers so next time the guess is a bit closer.
Make something new
Start from pure fog. Ask the brain "which way was the push?", step the other way, and repeat 100 times.
What just happened?
- At first the brain was brand new, so its dots stayed as random fog.
- It only ever practised one tiny job: guess the noise on shaken-up dots.
- The oops score dropped, and each checkpoint looked more like the real shape: fog → blob → shape.
- To make dots, it starts from fog and removes a little guessed noise at a time. That's diffusion!
- Picture-making AIs do exactly this, with millions of pixels instead of two numbers per dot.
For grown-ups: what's really going on
- Real training, in your browser. A plain-JavaScript MLP (no libraries) is trained live with backprop and Adam (batch 128, gradient clipping at 1).
- Objective. The DDPM "simple" loss from Ho et al. 2020: sample
x₀from the dataset,t ~ U{0..99},ε ~ 𝒩(0, I), formxₜ = √ᾱₜ x₀ + √(1−ᾱₜ) ε, and minimise‖ε − ε̂θ(xₜ, t)‖². Linear β schedule 1e-4 → 0.08, T = 100. - Network. Input is (x, y), Fourier features
sin/cos(k·x)for k ∈ {1, 2, 4, 8}, and a sinusoidal timestep embedding (34 numbers). Then 3 ReLU hidden layers of 32 / 64 / 128 units and 2 outputs (ε̂). The loss never reaches 0: even a perfect model can't know the exact noise, only its conditional mean, so the floor is well above 0, especially at high t. - Sampling. Ancestral DDPM:
xₜ₋₁ = (xₜ − βₜ/√(1−ᾱₜ) · ε̂) / √αₜ + σₜ z, with σₜ² = β̃ₜ. With fewer cleaning steps we use strided DDPM steps (a respaced schedule), as in Nichol & Dhariwal 2021. The arrows show−ε̂(the score direction, since∇log pₜ = −ε̂/√(1−ᾱₜ)). - Checkpoints. At steps 0, 25, 75, 150 … the weights are snapshotted and 500 samples are drawn. The bar is the overlap between the sample histogram and the data histogram, divided by what genuine data samples score.
- Inspired by tiny-diffusion. See the Diffusion Explainer (Polo Club) for the same process inside Stable Diffusion, where ε̂ comes from a U-Net working on image latents and is guided by text.