Real-time or batch: the first fork
Lesson 1 of 4 in Serving Predictions: Endpoints and Batch.
Nearly every Domain 3 scenario opens the same way: a trained model exists, and someone needs predictions from it. Your first move is never an instance type or a container — it is the inference-strategy fork the exam guide names: real-time or batch processing. Real-time keeps a persistent, managed endpoint waiting for requests; batch spins up a job, scores a dataset that already exists in Amazon S3, writes results back to S3, and shuts down. On top of those two strategies, SageMaker AI sells two more product options — serverless and asynchronous inference — and the exam expects you to recognize all four shapes.
| Option | Traffic shape | Payload · time | The tell in the stem |
|---|---|---|---|
Real-time | Sustained, latency-sensitive | Up to 25 MB · 60 s (streaming responses up to 8 min) | "Milliseconds", "interactive", "always available" |
Serverless | Intermittent, unpredictable, idle stretches | Up to 4 MB · 60 s | "Spiky", "no infrastructure to manage", "pay only while processing" |
Asynchronous | Individual large/slow requests that can queue | Up to 1 GB · up to 1 hour | "Queue", "large payload", "notify when done", "scale to zero" |
Batch transform | A whole dataset at once, no endpoint at all | GB-scale datasets · as long as the job runs | "Nightly", "score the entire table", "no persistent endpoint" |
Pick the serving mode
Interactive decision tree — outcomes:
- Real-time endpoint
Persistent instances behind an HTTPS endpoint, auto scaling on invocation load. You pay while it runs — idle included.
- Serverless inference
SageMaker AI manages capacity; it scales to zero between bursts, so you accept cold starts in exchange for paying only per use.
- Batch transform
A transform job reads from S3, scores everything, writes to S3, and terminates. No endpoint exists afterward — nothing bills overnight.
- Asynchronous inference
Requests queue; payloads up to 1 GB process for up to an hour; the instance count can scale to zero while the queue is empty.
One more reflex worth building now, because the cost module will lean on it: a real-time endpoint is provisioned — it bills every hour it exists, traffic or not. Batch jobs, serverless, and a scaled-to-zero asynchronous endpoint bill only around work. When a stem complains that "the endpoint sits idle overnight," it is steering you off real-time.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.