Architecture in Production: Design Choices Set Your Bill

The domain's production capstone: how attention's quadratic tax, GQA, MoE, and context length decisions surface as cost, latency, and capacity on real cloud hardware.

Content last verified 2026-09.

Lessons

1. Architecture Choices Meet the Bill
2. Attention's Quadratic Tax
3. GQA, MoE, and Context on Real Hardware

Sources

• Kaplan et al. (2020) — Scaling Laws for Neural Language Models
• Pope et al. (2022) — Efficiently Scaling Transformer Inference
• Ainslie et al. (2023) — GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
• Jiang et al. (2024) — Mixtral of Experts