Words → actions · Attention

Robo follows instructions

I get a little instruction, like go to the blue star. At first the words mean nothing to me. Press Train and watch me learn!

How often Robo picks the right thing (last 50 tries)

Checkpoints — Robo's brain saved along the way. Tap one to watch it try the same test.

They appear after 1, 25, 100, 300, 1000 and 3000 tries.

How it works

Words become numbers

Each word gets its own list of numbers. At first they're random.

Look at the words

For every thing on the grid, Robo asks: which words are about me? That's attention.

Pick and walk

The thing that matches the words best gets the biggest chance. Robo walks toward it.

Treat or oops

Right thing = reward. Wrong thing = oops. The word numbers get nudged after every try.

What just happened?

For grown-ups: what's really going on

A tiny attention-based policy, trained from scratch by policy gradient. No pretrained model, nothing downloaded.

  • Embeddings: each of the 9 vocabulary tokens has a learned vector e ∈ ℝ⁸.
  • Queries: each object's features (colour + shape one-hots) are projected to a query q = W·f. Attention of object k on token t is softmax_t(q_k·e_t), shown as the glow under each word chip.
  • Grounding: object scores m_k = Σ_t q_k·e_t → p = softmax(m). Move probabilities mix "moves that approach object k" weighted by p_k (plus 4% exploration).
  • Learning: REINFORCE with a running baseline. Reward +1 for the right object, −1 for a wrong one, −0.5 for running out of steps. Gradients flow through p into both the embeddings and W.
  • From ~33% (chance with 3 objects) it typically reaches ~95% on 200 held-out tasks within ~1000 episodes (well under a second of compute).
  • Real systems (instruction-following agents, vision-language-action models) use the same ingredients: token embeddings, attention to ground words in what the agent sees, and RL or imitation to learn actions.