Supervised Learning

Line Fitter

On hot days the ice-cream shop sells more cones. Find the one straight line that best follows the dots — then watch the robot learn it, one tiny step at a time.

Each dot is one day: how hot it was and how many cones we sold. Try dragging the two round handles on the line to make it fit the dots. Then press Robot, learn!

oopsie springs the line

Tap empty space to add a day · drag a dot to move it · tap a dot to remove it · drag the round handles to move the line

Snapshots while learning

Every time the robot reaches a step below, it takes a photo of its line. Tap a photo to look back in time.

Oopsie hills

Every spot on this map is a different line. Darker = smaller oopsie. The ball is our line rolling downhill; the flag is the very bottom. Tap the map to jump!

You vs Robot

Your best
–
Robot
–

Drag the handles first to set your best score, then let the robot try.

How it works

Guess a line

The robot starts with a totally random line. Probably a bad one!

Measure the oopsies

Each dot has a red spring pulling on the line. Long springs mean big mistakes.

Roll downhill

The springs tell the robot which way to nudge the line. It takes one small step that way.

Again and again

Step after step the springs shrink, until the line can't get any better.

What just happened?

For grown-ups: what's really going on

This is simple linear regression: the model is ŷ = w·x + b, fit by minimising the mean squared error MSE = (1/n) Σ (ŷᵢ − yᵢ)². The red springs are the residuals ŷᵢ − yᵢ; the "oopsie" number is MSE in cones².

  • The robot uses gradient descent: w ← w − η·∂MSE/∂w, b ← b − η·∂MSE/∂b, with ∂MSE/∂w = (2/n) Σ (ŷᵢ−yᵢ)xᵢ and ∂MSE/∂b = (2/n) Σ (ŷᵢ−yᵢ). Inputs are centred and scaled internally, which makes the loss bowl round and training fast.
  • "Oopsie hills" is the loss landscape over (w, b). For MSE it is a convex bowl, so gradient descent always finds the bottom if the learning rate η is small enough; above 2/λ_max of the Hessian it oscillates and diverges.
  • The checkpoint strip is the same idea as saving model checkpoints during training: snapshots of the parameters at chosen steps.
  • Linear regression also has an exact closed-form answer (ordinary least squares, the flag). Gradient descent matters because the same loop trains neural networks with millions of weights, where no formula exists.
  • The weird day shows squared error is sensitive to outliers.

Further reading: 3Blue1Brown, "Gradient descent, how neural networks learn" · An Introduction to Statistical Learning, ch. 3.