From Base Model to Assistant

A base model completes text; an assistant holds a conversation. The post-training map: what each stage adds, and what actually changes inside the model.

Content last verified 2026-09.

Lessons

1. The Gap: Completion Is Not Conversation
2. The Post-training Map
3. What Actually Changes

Sources

• Ouyang et al. (2022) — Training language models to follow instructions with human feedback (InstructGPT)
• Christiano et al. (2017) — Deep Reinforcement Learning from Human Preferences
• Rafailov et al. (2023) — Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Zhou et al. (2023) — LIMA: Less Is More for Alignment
• DeepSeek-AI (2025) — DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning