Artificial intelligence · Agents

Robot vacuum: rules or learning?

Two robot vacuums, one room. The blue one follows rules you write. The brown one has to learn. Press Play and see who cleans more!

Rules robot follows your rules
Learning robot learns from rewards

Dirt cleaned, round by round

Rules robotLearning robotThe room changed

Checkpoints — the learner's brain saved along the way. Tap one to watch it clean today's room.

How it works

Sense

The vacuum feels bumps, smells dirt a few squares away, and spots stairs and pets right in front.

Think

The blue robot looks down your list of rules. The brown robot looks up a score for each move in its notebook.

Act

Either way it can only do three things: go forward, turn left or turn right.

Feedback

Only the brown robot uses it: crumbs are treats, bumps and falls are ouches. It changes its notebook, and next time it does better.

What just happened?

For grown-ups: what's really going on

Both vacuums are intelligent agents in the sense of Russell & Norvig: they perceive their environment through sensors and act on it. They differ only in how percepts map to actions.

  • Rules robot: a simple reflex agent, a prioritised condition-action list. It is fast, transparent and needs no data, but it only handles situations its author imagined, and rules can fight each other (try "dirt on the left" together with "something blocks the way"). The "shake free" rule is a hack for loops, which a reflex agent with no memory cannot otherwise detect.
  • Learning robot: tabular Q-learning. State = (direction of nearest dirt within 3 squares or none, wall ahead/left/right, stairs ahead, pet ahead), which gives 160 states. Actions: forward, turn left, turn right. Update: Q(s,a) ← Q(s,a) + α [r + γ·max Q(s′,·) − Q(s,a)] with α = 0.2, γ = 0.9, ε-greedy exploration (default ε = 0.15). Rewards: +1 per dirt cleaned, −0.02 per move, −0.3 bump, −2 pet, −5 stairs.
  • Fair comparison: both robots live in identical rooms with the same dirt story each round (same random seed). A round is 160 moves.
  • Measured in Node (4 rooms, 60 rounds): naive rules about 11 dirt/round, the learner about 15, carefully written rules about 18. A learner needs no designer, but good hand-written rules can beat it in a world that never changes. Naive rules took 3–9 oopsies a round after stairs or a pet appeared and never improved. The learner fell to about 0–1 within a few rounds.
  • Further reading: Russell & Norvig, Artificial Intelligence: A Modern Approach (chapter 2, agents); Sutton & Barto, Reinforcement Learning.