Reinforcement Learning

Bird learns to fly

Help the bird fly through the gaps! Tap the picture (or press Space) to flap. Then switch to Robo learns and watch a bird teach itself.

Tap or Space to flap
Pipes passed, each trybars = each try · dark line = average of last 20
Checkpoints: tap one to replay that brainsame bird, different amounts of practice
Checkpoints appear here as Robo practises.

How it works

Look

The bird checks three things: how far the next pipe is, is the gap above or below me, and am I going up or falling.

Check the cheat-sheet

For every situation, its table has two scores: "wait" and "flap". It picks the bigger one.

Treats and ouches

A little treat for every moment alive, a bigger one for each pipe passed. A crash is a big ouch.

Fix the scores

After every flap or wait, it nudges that score toward what really happened next.

What just happened?

For grown-ups: what's really going on

This is tabular Q-learning on a Flappy-Bird clone, the same algorithm as the maze and pole lessons.

  • State: horizontal distance to the end of the next pipe (3 zones: near / mid / far), vertical offset of the gap centre (17 bins of 15 px), and vertical velocity (5 bins): just 255 cells. A deliberately coarse table generalises across similar situations, so it fills up fast: about 25 tries to average 10 pipes, versus about 120 with fine 20 px distance bins.
  • Actions: wait or flap. Reward: +1 per frame alive, +5 per pipe passed, −100 on a crash. α = 0.6, γ = 0.95, Q initialised to 0 and ties broken toward "wait".
  • Exploration: because Q starts at 0 and crashes are very negative, optimistic initialisation already explores well, so curiosity (ε) defaults to 0. Turn it up to see random flaps make it worse in the short term.
  • Offline in Node, the 100-try average reached 25–60 pipes after ~300 tries (well under a second of compute), and individual tries hit the 80+ pipe cap. Results vary between runs; tabular agents can also "forget" a little as they keep updating.
  • Deep RL replaces the table with a neural network that reads the raw pixels (as in DeepMind's Atari DQN, 2015).