Snake learns to eat
Steer the snake to the apples with the arrow keys. Don't bump the walls or your own tail! Then let a robot snake learn it from scratch.
Apples eaten, each trybars = each try · dark line = average of last 20
Checkpoints: tap one to replay that brainsame snake, different amounts of practice
Checkpoints appear here as Robo practises.
How it works
Feel
The snake only feels a few things: is there danger ahead, left or right, and which way the apple is.
Check the cheat-sheet
For each mix of feelings, its table has scores for "straight", "left" and "right". It picks the biggest.
Treats and ouches
An apple is a big treat. Bumping a wall or its tail is a big ouch. Wandering costs a tiny bit.
Fix the scores
After every move it nudges that score toward what really happened next. Hundreds of games later, it hunts apples.
What just happened?
- The first snakes crashed into walls almost right away. Their tables were empty.
- Each crash lowered the score for "that move, when I felt that". Each apple raised the score for the moves that led there.
- After a few hundred games, the robot snake eats 15–20 apples a game, all learned from treats and ouches.
- It still sometimes traps itself in its own tail. With only 7 feelings it can't plan far ahead. Smarter feelings (or a neural network) would help.
For grown-ups: what's really going on
Another tabular Q-learning agent, this time with a hand-designed, relative state.
- State (11 bits → at most 2,048 cells): danger straight / right / left (relative to heading), heading (4 values), and food up / down / left / right.
- Actions: straight, turn right, turn left. Relative actions mean the snake can never reverse into itself.
- Reward: +10 per apple, −10 for dying, −0.01 per step. A game also ends if the snake wanders too long without eating.
α = 0.3,γ = 0.9. - Exploration: with auto curiosity, ε decays from 0.1 to 0.005 over the first 300 games.
- Offline in Node over 3 runs: 50-game average ≈ 3 apples after 100 games, ≈ 8 after 200, ≈ 18 after 400. The greedy policy then averages 17–19 apples. Long snakes still die by coiling into themselves, because the state can't see the body beyond one cell.
- Deep RL versions (DQN) feed the whole grid into a neural network and can learn longer-range planning.