Queries, Keys, Values
Lesson 2 of 4 in Self-Attention: Every Token Looks at Every Token.
Picture a library. You walk in with a question — that is your query. Every book has an index card describing what it contains — those are the keys. And the contents of the books — what you actually take home — are the values. You compare your question against every index card, and the better a card matches, the more of that book’s content you take.
Self-attention runs this library for every token at once — and each token plays all three roles simultaneously. From its current Hidden state, each token computes three vectors by multiplying with three learned matrices: W_Q produces its query (what am I looking for?), W_K its key (what do I advertise to others?), and W_V its value (what content do I hand over if someone attends to me?). That triple is the Query, key, value (Q/K/V) scheme, and the three matrices are ordinary learned parameters, trained end to end with everything else.
| Query (Q) | Key (K) | Value (V) | |
|---|---|---|---|
Library analogy | The question you walk in with | The index card on each book | The book’s content you take away |
Question it answers | What am I looking for? | What do I advertise? | What do I contribute if chosen? |
Computed as | hidden state × | hidden state × | hidden state × |
Used for | Scored against every key | Scored against every query | Blended by the resulting weights |
Why three separate projections? Because what you match on is not what you hand over. The index card for a grammar book says ‘grammar reference’ — it does not contain the grammar rules themselves. A token like ‘animal’ may advertise ‘singular noun, recently mentioned’ (its key) while contributing rich semantic content about animals (its value). Collapse keys and values into one vector and the model must use the same representation for advertising and for delivering — a needless straitjacket.
And why learned? Because different lookups matter for different jobs, and nobody knows the full list in advance. Trained models turn out to contain heads that track subject–verb agreement, coreference, or matching brackets — behaviours discovered by the optimizer, not designed. One head is what you are studying here; the next module shows why real models run many heads side by side, each with its own W_Q, W_K, W_V operating in a smaller Head dimension-sized subspace of the d_model (model dimension)-wide Residual stream.
One token’s path through one attention head
- Token’s hidden state
A
d_model-dimensional vector — the token’s current representation, embeddings plus everything earlier layers added. - Query = x · W_Q
What this token is looking for.
- Keys = X · W_K
Every visible token advertises itself. X stacks all their hidden states.
- Values = X · W_V
The content each token will contribute if attended to.
- Score: query · every key
One dot product per visible token — high when query and key point the same way.
- Softmax → weights
Scores become positive weights that sum to 1.
- Weighted sum of values
Mostly the high-weight tokens’ values, a little of everything else.
- Attention output
Added back into the residual stream — the token’s representation, now context-enriched.
Key terms: Attention, Self-attention, Query, key, value (Q/K/V), Attention scores, Causal masking, Head dimension
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.