FM hosting: Bedrock, Custom Model Import, SageMaker
Lesson 1 of 4 in Deploying FMs, RAG, and Agents.
Foundation models fork the same way classic models did, one level up: who hosts the weights, and who owns the scaling? Amazon Bedrock answers "AWS does" — a serverless API over hosted FMs. SageMaker AI answers "you do" — real endpoints on GPU instances you pick, size, and scale. The C02 revision put this fork squarely in scope — evaluate and select FM deployment options — and it tests three Bedrock modes by name: on-demand, Provisioned Throughput, and Custom Model Import.
Bedrock on-demand
The default: call a hosted FM through the Bedrock API (InvokeModel / Converse), pay per token, provision nothing. Capacity is shared, so throughput is governed by service quotas — fine for most applications, variable for extreme ones. The stem tells: "no infrastructure", "pay per request", "occasional use".
Provisioned Throughput
Dedicated model capacity purchased in model units for a term — steady, guaranteed throughput for high-volume production. Two tells: "consistent/predictable high throughput", and serving models fine-tuned inside Bedrock — for those, Provisioned Throughput is required for every family except Amazon Nova, whose fine-tunes can alternatively use on-demand custom model deployment.
Custom Model Import
The 2026 blueprint's favorite child: you fine-tuned a model outside Bedrock (SageMaker AI, EC2, anywhere) on a supported open architecture — Llama, Mistral/Mixtral, and other listed families — and import the safetensors weights. Bedrock then serves it serverlessly with on-demand invocation, behind the same unified API as its native models. No instances, no capacity purchase.
SageMaker AI hosting
Full control: any architecture (including ones Bedrock cannot import), your container (AWS large-model-inference containers, or JumpStart's one-click FM deployments), your choice of GPU instance, your scaling policies. You take on GPU capacity planning in exchange for owning everything.
| Option | You manage | Billing shape | The tell in the stem |
|---|---|---|---|
Bedrock on-demand | Nothing | Per token | "No infrastructure", variable or light traffic |
Provisioned Throughput | Nothing (capacity is reserved) | Per model unit per term | "Guaranteed/steady high throughput"; the usual lane for Bedrock fine-tunes |
Custom Model Import | Nothing after import | On-demand invocation | Model fine-tuned elsewhere, wants Bedrock's API without instances |
SageMaker AI hosting | Instances, containers, scaling | Per instance-hour | Unsupported architecture, custom serving stack, GPU-level control |
Pick the FM hosting lane
Interactive decision tree — outcomes:
- Bedrock on-demand
Call the hosted FM through
InvokeModel/Converse, pay per token, provision nothing. Capacity is shared under service quotas — the default lane until traffic says otherwise. - Provisioned Throughput
Dedicated model units purchased for a term buy steady, guaranteed throughput — and this is the required lane for models fine-tuned inside Bedrock in every family except Amazon Nova, whose fine-tunes can alternatively deploy on demand.
- Custom Model Import
Import the externally fine-tuned safetensors weights; Bedrock serves them serverlessly with on-demand invocation, behind the same unified API as its native models. No instances, no capacity purchase.
- SageMaker AI hosting
A real endpoint: your container, your GPU instances, your scaling policies. Full control is the point — GPU capacity planning is the price.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.