Robot vacuum: rules or learning?
Two robot vacuums, one room. The blue one follows rules you write. The brown one has to learn. Press Play and see who cleans more!
Rules robot follows your rules
Learning robot learns from rewards
Dirt cleaned, round by round
Rules robotLearning robotThe room changed
Checkpoints — the learner's brain saved along the way. Tap one to watch it clean today's room.
How it works
Sense
The vacuum feels bumps, smells dirt a few squares away, and spots stairs and pets right in front.
Think
The blue robot looks down your list of rules. The brown robot looks up a score for each move in its notebook.
Act
Either way it can only do three things: go forward, turn left or turn right.
Feedback
Only the brown robot uses it: crumbs are treats, bumps and falls are ouches. It changes its notebook, and next time it does better.
What just happened?
- A rule-based robot is exactly as smart as its rules. The rules you wrote never got better by themselves.
- A learning robot starts clumsy and improves from treats and ouches, with nobody writing the rules.
- When the room changed, the learner adapted. The rules robot kept falling and bumping until a person added a rule.
- Both are AI. Only the learning one is machine learning.
For grown-ups: what's really going on
Both vacuums are intelligent agents in the sense of Russell & Norvig: they perceive their environment through sensors and act on it. They differ only in how percepts map to actions.
- Rules robot: a simple reflex agent, a prioritised condition-action list. It is fast, transparent and needs no data, but it only handles situations its author imagined, and rules can fight each other (try "dirt on the left" together with "something blocks the way"). The "shake free" rule is a hack for loops, which a reflex agent with no memory cannot otherwise detect.
- Learning robot: tabular Q-learning. State = (direction of nearest dirt within 3 squares or none, wall ahead/left/right, stairs ahead, pet ahead), which gives 160 states. Actions: forward, turn left, turn right. Update:
Q(s,a) ← Q(s,a) + α [r + γ·max Q(s′,·) − Q(s,a)]with α = 0.2, γ = 0.9, ε-greedy exploration (default ε = 0.15). Rewards: +1 per dirt cleaned, −0.02 per move, −0.3 bump, −2 pet, −5 stairs. - Fair comparison: both robots live in identical rooms with the same dirt story each round (same random seed). A round is 160 moves.
- Measured in Node (4 rooms, 60 rounds): naive rules about 11 dirt/round, the learner about 15, carefully written rules about 18. A learner needs no designer, but good hand-written rules can beat it in a world that never changes. Naive rules took 3–9 oopsies a round after stairs or a pet appeared and never improved. The learner fell to about 0–1 within a few rounds.
- Further reading: Russell & Norvig, Artificial Intelligence: A Modern Approach (chapter 2, agents); Sutton & Barto, Reinforcement Learning.