Robo follows instructions
I get a little instruction, like go to the blue star. At first the words mean nothing to me. Press Train and watch me learn!
How often Robo picks the right thing (last 50 tries)
Checkpoints — Robo's brain saved along the way. Tap one to watch it try the same test.
They appear after 1, 25, 100, 300, 1000 and 3000 tries.
How it works
Words become numbers
Each word gets its own list of numbers. At first they're random.
Look at the words
For every thing on the grid, Robo asks: which words are about me? That's attention.
Pick and walk
The thing that matches the words best gets the biggest chance. Robo walks toward it.
Treat or oops
Right thing = reward. Wrong thing = oops. The word numbers get nudged after every try.
What just happened?
- At first, words were just random numbers to Robo, so it wandered to any old thing.
- Every reward taught it a bit more about which words go with which colours and shapes.
- Soon it paid attention to the important words (blue, star) and ignored filler (go, to, the).
- Chatbots and robot helpers turn words into actions with this same attention trick, just much bigger.
For grown-ups: what's really going on
A tiny attention-based policy, trained from scratch by policy gradient. No pretrained model, nothing downloaded.
- Embeddings: each of the 9 vocabulary tokens has a learned vector
e ∈ ℝ⁸. - Queries: each object's features (colour + shape one-hots) are projected to a query
q = W·f. Attention of object k on token t issoftmax_t(q_k·e_t), shown as the glow under each word chip. - Grounding: object scores
m_k = Σ_t q_k·e_t→p = softmax(m). Move probabilities mix "moves that approach object k" weighted byp_k(plus 4% exploration). - Learning: REINFORCE with a running baseline. Reward +1 for the right object, −1 for a wrong one, −0.5 for running out of steps. Gradients flow through
pinto both the embeddings andW. - From ~33% (chance with 3 objects) it typically reaches ~95% on 200 held-out tasks within ~1000 episodes (well under a second of compute).
- Real systems (instruction-following agents, vision-language-action models) use the same ingredients: token embeddings, attention to ground words in what the agent sees, and RL or imitation to learn actions.