The Small Families
Lesson 2 of 3 in Small Language Models.
Who builds small? Nearly everyone — but in two postures, and telling them apart is a genuine map-reading skill.
Flagship families reach down. Most Open weights families ship a ladder of sizes, and the bottom rungs are their small models. Meta’s ladder runs from Llama 3.2’s 1B and 3B text models up through Llama 3.1’s “8 billion to 405 billion parameters.” Mistral’s current lineup lists Ministral 3 at 14B, 8B, and 3B. Google DeepMind pitches Gemma 4’s E2B and E4B for “mobile and IoT devices” beside 12B–31B siblings for “Frontier intelligence on personal computers.” OpenAI’s open-weight release pairs gpt-oss-120b with gpt-oss-20b — “21B parameters with 3.6B active parameters” — for lower latency and local or specialized use. Even specialist entries follow the pattern: the Qwen org page lists ASR models at 0.6B and 1.7B beside its far larger flagships. A reach-down model usually inherits its family’s tokenizer and chat format — and sometimes its license, which is exactly the thing you verify rather than assume.
Small-first builders start small. Microsoft’s Phi line leads with compactness: the one Phi card verified for this module calls itself “a compact open-weight multimodal reasoning model” — a 15B that is simultaneously Multimodal and a Reasoning model — and pointedly contrasts its recipe with “models that rely on substantially more training data and compute.” Hugging Face’s SmolLM3-3B comes from a team literally named Smol Models Research, and its card advertises what small-first culture values most: “Fully open model: open weights + full training details including public data mixture and training configs.”
| Small entry | Documented size(s) | License (as documented) | Documented trait |
|---|---|---|---|
Llama 3.2 — Meta | 1B and 3B text models; Llama 3.2 Vision at 11B and 90B | Vision card: “LLAMA 3.2 COMMUNITY LICENSE AGREEMENT,” described as “a custom, commercial license agreement”; distribution is gated — you “accept the license terms and acceptable use policy” before download. Text-size cards not read this run: verify per card | The bottom rungs of a ladder that reaches “8 billion to 405 billion parameters” in Llama 3.1 |
Ministral 3 — Mistral AI | 14B, 8B, and 3B | Apache 2.0, per the official models overview | The small rungs of Mistral’s current lineup, listed beside “Premier” commercial models |
Gemma small — Google DeepMind | Gemma 4 E2B & E4B; Gemma 3 at 1B and 4B (family spans 1B–27B) | Gemma Terms of Use — with a Prohibited Use Policy and Google’s right to “remotely restrict violating usage”; the terms page notes Gemma 4 has a separate Apache 2 license | E-variants pitched for “mobile and IoT devices”; the Gemma 3 card states its “4B model was trained with 4 trillion tokens and 1B with 2 trillion tokens” |
Qwen3-0.6B — Qwen Team | “Number of Parameters: 0.6B” (“Non-Embedding: 0.44B”) | apache-2.0, as listed on the model card | “Context Length: 32,768”; switching between thinking and non-thinking modes in one model; “100+ languages and dialects” |
Phi — Microsoft | Phi-4-Reasoning-Vision-15B verified this run: 15B, 16,384-token context | MIT — “License: mit” on that card; verified for that model only | “a compact open-weight multimodal reasoning model” with hybrid modes: chain-of-thought for math and science, direct answers for perception tasks |
SmolLM3-3B — Hugging Face | “a 3B parameter language model,” with a base variant and quantized versions linked | Apache 2.0, as listed on the model card | “dual mode reasoning, 6 languages and long context”; fully open training details, including the public data mixture |
gpt-oss-20b — OpenAI | “21B parameters with 3.6B active parameters” (mixture-of-experts) | Apache 2.0 — the card calls it a “Permissive Apache 2.0 license” | The smaller of a two-variant release, positioned for lower latency and local or specialized use; adjustable reasoning effort |
Read the license column twice — it is the lesson. The small end compresses the whole open-landscape license spectrum into one table: permissive grants (Apache 2.0 for Ministral 3, Qwen3-0.6B, SmolLM3-3B, and gpt-oss-20b; MIT on the verified Phi card), custom terms with use policies and remote-restriction rights (Gemma Terms of Use), and a Community license with Gated distribution plus naming and attribution requirements (Llama). Parameter count buys you nothing here: the license check is the same size at every model size. And the hedges are not decoration — a license field binds one artifact, so “verified for that model only” is what careful reading looks like. Families genuinely mix terms across versions: the Gemma terms page itself notes that Gemma 4 carries a separate Apache 2 license.
Notice also what “small” means: nothing absolute. gpt-oss-20b is the small variant of its release; Qwen3-0.6B is thirty-five times smaller than that. The word only makes sense relative to a family’s ladder and your hardware floor — read it that way on every Model card. Mixture-of-experts adds a wrinkle worth catching: gpt-oss-20b’s “3.6B active parameters” describes per-token compute, while the 21B total sets the memory footprint. Cards that state both numbers are answering two different questions — speed and fit — and you need both answers.
Context windows are card-stated facts too, and cards state them differently. Qwen3-0.6B’s card gives a single spec line — “Context Length: 32,768” — while SmolLM3’s card says it was “Trained on 64k context and supports up to 128k tokens using YARN extrapolation.” Native versus extrapolated is a real distinction for your Context window planning; a careful reader keeps the two numbers apart because the cards do.
Interactive sorting exercise: Sort each documented small model by how it enters the landscape: as the bottom rung of a bigger family’s ladder, or from a builder that starts small on purpose. Classifications follow the model cards quoted in this module.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.