Three ways to host a model
Lesson 1 of 3 in Model Deployment and Throughput Strategies.
Every FM request terminates somewhere, and there are exactly three somewheres: Bedrock on-demand, Bedrock Provisioned Throughput, and SageMaker AI endpoints. Knowing which one a scenario is describing — and what each costs you in flexibility — is most of this topic.
Bedrock on-demand is the default: serverless, pay-per-token, shared capacity governed by account quotas — ideal for unpredictable traffic, and the lane your Lambda functions call for on-demand invocation. Bedrock Provisioned Throughput replaces shared capacity with dedicated capacity bought in Model Units (MUs) — each MU guarantees a specific number of input tokens processed and output tokens generated per minute — billed hourly, with no-commitment, 1-month, and 6-month terms (longer commitment, deeper discount). And one rule the exam loves: models you customize in Bedrock require Provisioned Throughput to serve (weights imported via Custom Model Import are the serverless exception).
The third lane leaves Bedrock entirely: SageMaker AI endpoints, where you choose the instances, bring any weights, and own the serving stack — the lane for open-weights models Bedrock does not host, strict latency engineering, or container-level control. Hybrid architectures mix lanes deliberately: Bedrock on-demand for spiky consumer traffic, a provisioned fine-tune for the steady enterprise workload, a SageMaker endpoint for the exotic model.
| Bedrock on-demand | Bedrock Provisioned Throughput | SageMaker AI endpoint | |
|---|---|---|---|
You pay for | Tokens processed (zero when idle) | Model Units per hour, commitment-discounted — whether or not traffic arrives | Instances per hour while the endpoint is up |
Capacity | Shared pool behind account quotas — throttling (429s) under burst is expected behavior | Dedicated: each MU guarantees input + output tokens per minute | Whatever your instance fleet and auto scaling provide |
Models | Hosted catalog (plus Custom Model Import weights) | Base models and — critically — Bedrock-customized (fine-tuned) models | Anything you can containerize: open weights, custom architectures |
Exam cues | "Unpredictable traffic", "minimal operational overhead", "pay only for use" | "Guaranteed/consistent throughput", "fine-tuned Bedrock model in production", "throttling at peak" | "Full control", "custom container", "model not available in Bedrock" |
Pick the hosting lane
Interactive decision tree — outcomes:
- Bedrock on-demand
Serverless, per-token, zero idle cost. Pair with SDK retries and backoff for the quota-shaped 429s — resilience patterns live in FM APIs and streaming.
- Provisioned Throughput (sized in MUs)
Dedicated tokens-per-minute capacity, billed hourly, discounted by 1- or 6-month commitment. Size MUs from measured input/output token rates, not guesses.
- Provisioned Throughput — not optional
Serving a model customized in Bedrock requires purchasing Provisioned Throughput. This is the exam's favorite PT fact: fine-tuned + production = PT.
- Custom Model Import
Import compatible safetensors weights and invoke them on demand behind the standard Bedrock API — the serverless path for externally trained models.
- SageMaker AI endpoint
Your instances, your container, your serving stack — maximum control, maximum ownership. The next lesson covers what sizing one for an LLM actually means.
Key terms: Provisioned Throughput (Bedrock), Custom Model Import, Amazon Bedrock
Interactive checkpoint quiz (3 questions) — open this page in a browser to take it.