Where data lives: S3, EBS, EFS, and friends
Lesson 1 of 4 in The Supporting Cast: Data, Compute, and Pipes.
Amazon S3 is the center of gravity. Training data lands there, training jobs read from there, model artifacts (model.tar.gz) are written back there, Athena queries it in place, Knowledge Bases ingest documents from it, and batch transform reads and writes it. When an exam scenario does not say where data lives, it lives in S3.
S3 is not one shelf but a ladder of storage classes: Standard for hot data, Intelligent-Tiering when access patterns are unknown (it moves objects between tiers automatically), Standard-IA and One Zone-IA for infrequent access, and the Glacier trio for archives — Instant Retrieval (milliseconds), Flexible Retrieval (minutes to hours), Deep Archive (hours, the coldest). A high-performance single-AZ class (S3 Express One Zone) exists for latency-critical workloads. Lifecycle rules move objects down the ladder on a schedule — the standard answer to "old training runs are expensive to keep".
| Service | What it is | Attach model | ML pattern |
|---|---|---|---|
Amazon S3 | Object storage, API-accessed from anywhere | No mounting — requests over HTTPS | The default: training jobs read S3 in File mode (copy before training) or stream it with FastFile/Pipe mode |
Amazon EBS | Block volumes — a network disk | Attached to one instance at a time | The scratch disk under notebooks and training containers; never shared storage |
Amazon EFS | Elastic NFS file system | Mounted by many instances concurrently | Shared home directories in SageMaker Studio; a mountable training data source when many jobs need the same files without copies |
FSx for Lustre | High-performance parallel file system | Mounted by many instances, built for throughput | Links to an S3 bucket and serves it at high throughput — the pick when large distributed training would otherwise wait on S3 copies |
Which S3 class for which ML artifact?
Active training data and features: Standard. Unknown or shifting access: Intelligent-Tiering (no retrieval logic to write). Older artifacts you may need quickly: Standard-IA or Glacier Instant Retrieval. Compliance archives of past datasets and models: Glacier Flexible Retrieval / Deep Archive.
What breaks if you archive carelessly?
A training job pointed at objects in Glacier Flexible Retrieval or Deep Archive cannot read them until they are restored — archive tiers are for data you can afford to wait for. Never put next week's training data behind a retrieval delay.
EBS vs EFS in one sentence
EBS is one instance's disk; EFS is a network drive many instances mount at once. If the scenario says 'shared across training instances', EBS is already eliminated.
Storage Gateway, in one line
The hybrid bridge: on-premises applications keep speaking file/volume/tape protocols while the bytes land in AWS storage — one way on-prem data reaches your S3 training lake.
Interactive sorting exercise: Sort each workload onto the storage service the exam wants.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.