Staying Current

Lesson 3 of 3 in In Production: Reading a Model Card and Choosing a Model.

The landscape will not hold still, and the wrong response is to move every time it does. Chasing each Model release burns evaluation budget, destabilizes prompts tuned to the incumbent, and replaces engineering judgment with novelty. The right response is cadence: a scheduled rhythm of re-verification, with a short list of hard triggers that can interrupt it.

The cadence rests on a habit this site practices on itself: everything carries an asOf date. Every module here wears a “Last verified” stamp, and every fact table records the day its sources were read — the model facts in this domain were verified 2026-09-16 against the cards and license pages they cite. The stamp is not decoration; it is an expiry-aware claim. “Llama 4’s license requires attribution” is not a timeless truth — it is a fact as of a date, awaiting re-verification. Your own artifacts deserve the same: a requirements one-pager with a date, a family map with a date, harness results with a date.

Between quarterly re-verifications, the input stream is a release log — the pattern this site’s Release Log implements: a dated, append-only record of what changed in the landscape, scanned on your schedule rather than pushed into your attention. And on the loss side of the ledger, Deprecation watching: models leave the landscape as surely as they enter it. Vendors’ own docs say so plainly — Mistral’s official models overview keeps a deprecated-and-retired table where once-headline names like Mixtral and Pixtral now live (docs), and OpenAI’s model docs page lists a realtime model as deprecated in the current lineup (docs). When a model changes or retires under you, the playbook is the regression machinery from Production Evals — your golden set is what makes a migration a measurement instead of a leap.

Quarterly: re-verify the licenses you depend on

License pages are living documents — the Gemma Terms of Use page carries its own last-modified date (the version read for this site’s fact file was modified April 1, 2026), and one revision can change obligations or add a separate regime for a new generation. Re-read the page itself, not your notes about the page, for every license your product ships under.

Quarterly: scan the deprecation and retirement tables

Every vendor with a hosted lineup maintains lifecycle documentation; every model you run belongs on a watchlist against it. A Deprecation notice with a long runway is a gift — it converts a future emergency into a scheduled shortlist re-run.

Quarterly: re-run the harness on the incumbent plus one challenger

Two birds: Regression testing catches the incumbent drifting under silent updates, and the challenger run keeps your sense of the frontier calibrated by your numbers rather than by launch-day noise. One challenger per quarter is enough — the point is calibration, not tournament.

Quarterly: re-stamp the one-pager

Requirements drift too: a new input type arrives, context needs grow, legal’s posture on community licenses shifts, unit economics move the cost ceiling. Re-date the one-pager even when nothing changed — “re-verified, unchanged, 2026-Q4” is itself a decision record.

Anytime: the hard triggers that interrupt the cadence

Three events justify acting off-schedule: a deprecation or retirement notice for a model you run; a regression on your golden set in production; and a requirements change that invalidates the current choice (new modality, new data boundary, new ceiling). Note what is not on the list: a launch announcement, a leaderboard reshuffle, a competitor’s press release. Those go to the release log, and wait for the quarter.

This is where the academy closes, so take stock of what you now carry. Ten domains, and none of them was really about memorizing its subject — each installed one instrument you will still be using when every model named on this site has retired.

The mechanism layers (L-01–L-04) taught what the machine is: next-token prediction, the transformer that computes it, the pre-training that fills it with the world, the post-training that makes it usable. That is why nothing in a launch announcement mystifies you: you know what kind of object shipped. The meter layers — inference and serving (L-05), the cloud platforms (L-09) — taught where latency and cost are manufactured, so a card’s active-parameter count or context line converts in your head to serving arithmetic. Adaptation (L-06) taught the ladder from prompting to Fine-tuning, so “the model can’t do it” became “which rung fixes it”. Evidence (L-07) taught you to trust nothing you didn’t measure — the Eval harness that this module’s protocol runs. Risk (L-08) taught what fails and what an attacker does with it. And this domain, the map (L-10), taught you to read the territory those instruments operate in: families, weights, licenses, cards.

Watch the loop close on any future release. The map tells you where it sits and what the license permits. The mechanism tells you what plausibly changed. The meter prices it. Adaptation says what it would take to fit your task. Evidence decides — your golden set, your bar. Risk gates the rollout. That loop is the skill this site exists to install: not knowing today’s answer, but owning the procedure that produces the answer on any given day.

The academy as your toolkit — the loop you now own

  1. The map — L-10 · Model Landscape

    Families, open vs closed weights, licenses, model cards — the navigation layer. Read quarterly, never memorized: every fact here wears an asOf date, and the reading skill outlives every map it reads.

  2. Risk — L-08 · Security & Risk

    What fails and what attackers do with it: hallucination, injection, leakage, red teaming. The gate a chosen model passes through before users meet it.

  3. Evidence — L-07 · Evaluation

    Benchmarks and their limits, contamination, judges — and the harness that turns model choice from taste into measurement. The step every other layer defers to.

  4. Adaptation — L-06 · Adapting LLMs

    The ladder from prompting through RAG to fine-tuning and distillation — how a general model becomes your model, and how to pick the cheapest rung that works.

  5. The meter — L-05 · Inference & Serving · L-09 · LLMs on the Cloud

    Where cost and latency are manufactured: sampling, the KV cache, quantization, batching — and the platforms, quotas, and deployment shapes that meter it all in production.

  6. Mechanism — L-01–L-04 · Foundations → Post-training

    The machine itself: next-token prediction, the transformer, pre-training data and scaling, post-training and alignment. Everything above stands on knowing what kind of object an Large language model (LLM) is.

In production

Staying current is a production discipline on the clouds too: every platform versions its models, publishes lifecycles, and retires what you depend on — on its schedule, not yours.

AWS

Amazon Bedrock publishes model lifecycle states and end-of-life documentation for the models it hosts — treat that page like a dependency board for your stack. Pin exact model identifiers in configuration rather than accepting whatever “latest” resolves to, watch the platform’s announcements as your release log, and make your regression suite the gate any lifecycle-forced migration must pass before traffic moves.

Azure

Azure AI Foundry documents model deprecations and retirements with dates and recommended replacements, and deployments can be configured to auto-upgrade to newer versions. Decide that setting explicitly: auto-upgrade trades control for currency, and either choice should route through your golden set — an upgrade you did not evaluate is a model change you did not test.

Google Cloud

Vertex AI versions its foundation models and publishes availability and discontinuation information per version, while Model Garden entries evolve as upstream families release and retire. Your quarterly ritual on GCP includes re-reading the entries and version pages you depend on — and because open models keep their upstream licenses, re-verifying those license pages is part of the same pass.

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.