Post-training & Alignment

SFT, RLHF, DPO, and reasoning training — how a base model becomes an assistant

• From Base Model to Assistant — A base model completes text; an assistant holds a conversation. The post-training map: what each stage adds, and what actually changes inside the model. (3 lessons, 40 min)
• Supervised Fine-Tuning — Teaching by demonstration: instruction data, chat templates, the training recipe — and why data quality beats quantity. (4 lessons, 45 min)
• RLHF: Learning from Preferences — Reward models trained on human comparisons, policy optimization against them, and the failure mode that haunts the whole method: reward hacking. (4 lessons, 50 min)
• DPO and the Direct Methods — Direct preference optimization skips the reward model entirely. The trick, the math, the variants — and when classic RLHF still wins. (3 lessons, 45 min)
• AI Feedback: RLAIF and Constitutional Methods — When the annotator is a model: AI-generated preferences, constitutions and critique loops, and the honest limits of self-supervision. (3 lessons, 40 min)
• RL for Reasoning Models — Verifiable rewards changed the game: training models to think longer on math and code, what reasoning training buys, and what it costs at inference time. (3 lessons, 45 min)
• In Production: What Alignment Does and Doesn’t Guarantee — The domain’s production capstone: the behavior profile you inherit with every model, the gap between aligned and safe-for-your-app, and how to test before you trust. (3 lessons, 40 min)