One Token's Journey

Lesson 2 of 4 in One Token's Journey: The Full Forward Pass.

Take a five-token prompt: “The cat sat on the”. The model’s only job is Next-token prediction — produce a probability for every token that could come next. Follow the final token, “ the”, up the stack, because it is the token whose top-of-stack Hidden state will make the prediction.

The journey starts with no neural network at all. The Tokenizer (Tokenization module) splits the text into tokens using its learned subword vocabulary — here, five common words that each map to a single id. Then the first learned Weights appear: each id selects one row of the Embedding matrix (Embeddings module), and our token becomes a vector of width d_model (model dimension). At this point the vector for “ the” is generic — the same row every occurrence of “ the” gets. Position information (Positional Encoding module) is what lets the model tell this “ the” at position 5 apart from the one at position 1: added directly to the embedding in older designs, or applied as a rotation inside every attention layer with Rotary position embedding (RoPE).

One token’s journey, end to end

  1. Prompt: “The cat sat on the”
  2. Tokenizer

    The text becomes five tokens, each an integer id from the subword vocabulary. Pure lookup — no network yet.

  3. Embedding lookup

    Each id selects one row of the embedding matrix: a vector of width d_model per token.

  4. Position information

    Added to the embeddings (learned or sinusoidal schemes) or applied inside each attention layer (RoPE) — either way, position 5 becomes distinguishable from position 1.

  5. Block 1 — attention

    The token’s query is compared against the keys of every position; causal masking hides positions after it. The weighted mix of values — context — is added into its residual stream.

  6. Block 1 — FFN

    A feed-forward network transforms this position on its own, no cross-token communication, and adds its result back into the stream.

  7. Blocks 2 … N

    The same two moves repeat, block after block. Each pass refines the hidden state: syntax early, meaning and prediction-relevant features later.

  8. Final norm

    One last LayerNorm or RMSNorm settles the residual stream before readout.

  9. Unembedding

    The last position’s vector is multiplied by the unembedding matrix: one raw score — a logit — for every vocabulary entry.

  10. Softmax

    Exponentiate and normalize: logits become probabilities that sum to 1.

  11. Next-token probabilities

    For this prompt: “mat”, “floor”, “couch”… (illustrative). A sampler picks one; the journey repeats.

Now the vector enters block 1, and the two moves you know from Self-Attention and Inside the Block begin to alternate. In the attention step, our token’s query is compared against the keys of all five positions (Query, key, value (Q/K/V)); Causal masking guarantees it can only look at itself and the four tokens before it. Every Attention head does this in parallel with its own notion of relevance — one head may fetch “cat” (who is sitting?), another “on” (a location is coming). The weighted mix of values is added into the token’s Residual stream. Then the Feed-forward network (FFN) step transforms the position by itself — no communication with other tokens — and adds its result back too.

That pair of moves repeats N times. Nothing else changes: same width, same stream, same recipe. (In a Mixture of experts (MoE) model, the FFN step is swapped for a Router choosing a few experts per token — the journey is otherwise identical.) What changes is the content of the vector. By the top of the stack, position 5 no longer holds “the meaning of the word ‘the’” — attention has folded in the cat, the sitting, and the preposition, and the FFNs have transformed that mixture toward one thing: what comes next.

One journey produces one token. To write a sentence, the model samples a token from the distribution, appends it to the input, and runs the journey again — and again, once per generated token. Two useful notes for later: the keys and values of earlier positions do not change between these runs, which is why serving systems cache them (the KV cache — mechanics in the Inference & Serving domain); and everything an assistant appears to “do” — answer, reason step by step, call a tool — is this loop. What happens when systems act on those outputs belongs to the sister AI Agent Academy; here we stay inside the model.

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.