Resilient model architectures: provider switches and cross-Region inference
Lesson 3 of 4 in Designing GenAI Solutions and Selecting Models.
Model abstraction and resilience answer two different fears. Provider risk: the best model next quarter may not be this quarter's — so route every model call through one abstraction (API Gateway + Lambda), keep the active model or inference profile ID in AWS AppConfig, and marshal requests with the Converse API's uniform shape. Switching providers becomes a configuration deployment — validated, gradual, rollback-able — with no code modifications, the literal phrase the AIP-C01 exam guide uses.
Capacity risk: a Region throttles under burst or a model has limited regional availability. Cross-Region inference answers it natively — invoke a geographic profile (us./eu. prefixed) and Bedrock routes each request to healthy capacity within the geography, by availability, latency, and demand; global profiles widen routing to every Region offering the model. Around both risks sit circuit breakers (Step Functions + DynamoDB state: stop hammering a failing dependency, fail fast to a fallback) and graceful degradation (a smaller model, cached answers, retrieval-only mode — reduced service beats no service).
The circuit breaker, concretely
A Step Functions state machine fronts the model call. Before invoking, it checks breaker state in DynamoDB: closed → call normally; repeated failures past a threshold flip it open → requests fail fast to the degradation branch instead of queueing against a sick dependency; after a cooldown, half-open sends trial requests, and success closes the breaker. The exam phrase is literal: "AWS Step Functions circuit breaker patterns".
The graceful-degradation ladder
Decide the ladder before the outage: primary model → smaller/alternate fallback model → cached or templated answers → retrieval-only mode ("here are the relevant documents") → honest error with retry guidance. Each rung keeps more users served than the one below it. Degradation is designed, not improvised.
Provider-switch day, in practice
Because every call already flows through the adapter and the Converse API, the switch is: create the new model/profile config in AppConfig → deploy it gradually (AppConfig supports measured rollouts and automatic rollback on alarms) → run the prompt regression suite against the new model → complete or roll back. The stems reward this over "update the Lambda code and redeploy."
Geographic vs global profiles
Geographic profiles (us., eu., …) route only within their geography — the default when residency or latency bounds matter. Global profiles route to any commercial Region offering the model for maximum throughput headroom, at the price of geographic predictability. Pick by whichever constraint the stem states.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.