Self-attention

Where to look

Sentences are tricky! In "The cat chased the mouse because it was hungry", who is it? To work that out I shine a spotlight on the right words. You try first!

Play: you are the spotlight

Who is "it"? Tap one word (or two, if you're not sure) to aim your spotlight. Round 1 of 6

Watch Robo learn where to look

Test sentences Robo has never seen

Picks the right word Attention on the right word A challenge

Challenges

A sneaky extra noun not tried

Add a distractor, like "near the fence". The right answer is still an animal. Can Robo ignore it?

Swap one word not tried

Change hungry to scared (or big to small). The answer should flip to the other noun!

Hide the answer not tried

Cover up the right noun. Does Robo's spotlight move to the other noun?

Checkpoints — Robo's brain saved along the way. Tap one to see where it looked on the sentence above.

They appear after 0, 10, 30, 100, 250 and 600 rounds.

How it works

Words become numbers

Every word gets a list of numbers, plus a little number for where it sits in the sentence.

Ask and answer

The word it makes a question. Every other word holds up a label saying what it is.

Shine the spotlight

Questions that match labels get bright light. All the light adds up to 100%.

Nudge and repeat

Light on the right noun? Great. If not, the numbers get nudged so next time it's brighter.

What just happened?

For grown-ups: what's really going on

This is scaled dot-product attention, trained from scratch with analytic gradients and Adam. No pretrained model, nothing downloaded.

  • Query, key, value: each token vector is x = embedding + position. The asker's query is q = Wqc, every token has a key kj = Wkxj. Here the output we care about is the weights themselves; a full Transformer also mixes value vectors with them.
  • Weights: a = softmax(q·kj / √d) with d = 12. The softmax makes them sum to 1. Dividing by √d keeps scores from growing too big and saturating. Hidden words get a score of −∞, so their weight is exactly 0.
  • Loss: −log of the weight on the correct noun. Gradients flow into Wq, Wk, the word embeddings and the position vectors. They are checked against finite differences (relative error about 1e-8).
  • Positional information: attention itself ignores word order, so a learned vector per position is added. Here that is how "first noun" and "second noun" differ.
  • Simplification: in a real model a first layer lets "it" absorb the nearby clue word ("was hungry"). We feed c = eit + eclue directly, and only the "it" row is trained, so other rows in the heatmap are mostly noise.
  • Results: trained on 600 templated sentences, tested on 300 unseen ones, from about 40% (guessing) to 100% in roughly 100 rounds of 6 sentences. The weight on the correct noun goes from about 10% (1 / sentence length) to above 95%.
  • Real LLMs: use multi-head attention (many query/key/value sets looking for different things, like grammar, who-did-what, or nearby words), a causal mask so a word only sees earlier ones, and stack dozens of such layers, each followed by a small feed-forward network.