Diagnosing GenAI failures
Lesson 1 of 4 in Troubleshooting GenAI — and Exam Readiness.
GenAI systems fail in two registers, and the professional move is triaging them in order. Deterministic failures arrive as coded exceptions — throttles, validation rejections, timeouts — and are solved with error logging, request validation, and response analysis. Stochastic failures are quieter: the API returns 200 and the content is wrong, truncated, or weird. Chase the loud ones first; they are faster to kill and they frequently explain the quiet ones — the "flaky quality" that turns out to be silent truncation.
Read the error class before the error: a 4xx means your request (shape, size, permissions, rate); timeouts and 5xx mean the path or the model. The table below is the Bedrock triage card.
| Error | What it actually means | First move |
|---|---|---|
ValidationException — "Input is too long for requested model" | Context-window overflow: history + retrieved chunks + instructions exceeded the model's token capacity | Log token counts per prompt segment; shrink with dynamic chunking, history summarization, retrieval top-k budgets |
ThrottlingException (HTTP 429) | On-demand quota saturated — comes in two flavors: "too many requests" (request rate) and "too many tokens" (token rate) | Exponential backoff with jitter on every client; for sustained load, provisioned throughput or cross-Region inference to spread bursts |
AccessDeniedException | The identity cannot invoke this model — model access not enabled in this Region, or an IAM policy gap | Verify model access is granted in the calling Region and the execution role allows the invoke action on that model |
ResourceNotFoundException | The model ID or inference profile does not exist where you are calling — typo, wrong Region, wrong profile ARN | Diff the identifier in the failing request against the console/CLI listing for that Region |
ModelTimeoutException | Generation ran past the model timeout — usually very long outputs on a synchronous call | Stream the response instead of waiting for completion; cap output tokens; trim the prompt |
ServiceQuotaExceededException | A hard account quota was hit (concurrency, job counts), distinct from momentary throttling | Request a quota increase; until then, queue and spread the workload |
Past errors and truncation sits the slipperiest class: prompt problems. The professional posture is treating prompts like code. Versions live in Bedrock Prompt Management, so "it got worse" becomes version comparison — run v11 and v12 over the same golden set and diff the scores, instead of arguing from memory. Structured outputs get schema validation at the boundary, so a format regression is caught as a validation failure the moment it happens, not three services downstream. And every invocation carries its prompt-version id through the Prompt observability pipeline — X-Ray spans for per-hop timing, CloudWatch Logs for assembled prompts — so "which change broke Tuesday" is a filter expression. Refinement then runs as a loop: hypothesis → edit → re-evaluate on the same set → compare, never "tweak and vibe-check".
Signature: the answer stops mid-sentence
That is the output side — the max-tokens ceiling ended generation early (or a stop sequence misfired). Raise the output cap or stream. Do not confuse it with input overflow: input truncation produces complete, confident answers about an incomplete prompt.
Signature: it ignores the end of long inputs
Input-side overflow (silent truncation) — or, if sizes are within budget, positional weakness in long contexts. Telemetry splits them: token counts at the max mean truncation; healthy counts with middle-of-document misses point at prompt structure (move critical content toward the edges, retrieve less noise).
Signature: valid JSON yesterday, prose today
A prompt version drifted or the model version moved underneath you. Schema validation catches it at the boundary; version pinning plus a regression run tells you which change did it. The exam phrase is "schema validation to detect format inconsistencies".
Signature: works in the console, fails in the app
A request-shape problem: wrong parameter casing, missing inference profile, different Region, a mangled body. Log the exact failing request and validate it against the API schema — request validation is the first tool the exam guide names.
Signature: fine all day, bursts of 429s at the top of the hour
Scheduled jobs colliding with user traffic on on-demand quotas. Backoff with jitter absorbs the spikes; move the batch work to off-peak or to provisioned throughput; cross-Region inference spreads what remains.
Key terms: Context-window overflow, Prompt observability, Bedrock Prompt Management, AWS X-Ray
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.