Conversation history and state design
Lesson 2 of 3 in Prompt Engineering and Governance.
Foundation models are stateless: the "conversation" your users experience is a data structure your application maintains and replays. The standard design is DynamoDB for conversation history — a session table (partition key sessionId, sort key turn number or timestamp) written after every exchange and read before the next, with TTL expiring sessions on your retention policy. Around it, Step Functions drives clarification workflows (the ask-a-follow-up-question loop when a request is ambiguous), and Amazon Comprehend supplies intent recognition for routing turns to the right handler or prompt.
The engineering tension is the context budget: history grows every turn, tokens cost money, and windows are finite. Production designs send a window of recent turns, summarize the older past, and keep the full transcript in the table — the model sees what it needs, the record keeps everything.
Clarification workflows (Step Functions)
When a request is ambiguous ("cancel my subscription" — which of three?), the right move is a follow-up question, not a guess. Step Functions models the loop as states: detect ambiguity → ask → await the answer → proceed — the AIP-C01 exam guide's "Step Functions for clarification workflows". The wait state is the point: conversations have gaps that code with a 29-second timeout cannot hold.
Intent recognition (Comprehend)
Classify the turn before you spend FM tokens on it: Comprehend custom classification routes "billing question" vs "technical fault" vs "cancellation" to the right prompt template, knowledge-base scope, or human queue. Cheap classification in front of expensive generation.
Windowing vs summarization
Windowing sends the last N turns verbatim — simple, loses old context sharply. Summarization compresses older turns into a rolling synopsis the window carries — smoother memory, costs a summarization call. Most production designs run both: synopsis + recent verbatim turns. (Retrieval over the transcript is the heavier third option when sessions span weeks.)
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.