Containers for ML: ECR, ECS, EKS
Lesson 2 of 4 in Provisioning ML Infrastructure as Code.
Every model you deploy on AWS runs in a container, and every container lives in Amazon ECR — including the ones SageMaker AI pulls when it stands up an endpoint. That makes ECR two things at once: the registry for your own images, and the delivery mechanism for AWS's prebuilt framework images. The real exam question is one level up: when a team serves models on raw containers (ECS or EKS), and when SageMaker hosting earns its keep.
Where serving images come from
Three tiers, all resolving to ECR: AWS prebuilt images (Deep Learning Containers for PyTorch, TensorFlow, XGBoost, sklearn, and large-model inference) cover most cases; extended prebuilt images add your dependencies on top; fully custom images exist for exotic stacks — in C02 you only need the concept, not the container contract.
ECR beyond storage
Image scanning flags known CVEs in layers; tag immutability stops :prod from silently meaning something new — version pins that matter again in the MLOps module; lifecycle policies expire stale images.
ECS vs EKS in one breath
ECS is AWS's own orchestrator — simpler, deeply integrated, and with Fargate it runs containers with no instances to manage. EKS is managed Kubernetes — maximum control and portability, GPU scheduling, and the natural home when the organization already lives in the K8s ecosystem.
| ECS (+ Fargate) | EKS | SageMaker AI hosting | |
|---|---|---|---|
What you manage | Task definitions, services; no instances on Fargate | The cluster's workloads; K8s manifests, node groups | A model, a config, an endpoint |
ML features | None built in — you add scaling, routing, monitoring | What you install (KServe, Kubeflow…) | Variants, multi-model, invocation-metric auto scaling, deployment guardrails, Model Monitor hooks |
The tell | "Simplest managed containers", existing ECS estate | "Standardized on Kubernetes", portability, GPU scheduling control | "Managed ML serving", A/B tests, drift monitoring — without building any of it |
Interactive sorting exercise: Each card names a workload or requirement. Drop it on the service that owns it.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.