Ingestion & Transformation

Streams and batches in, Glue and Spark and EMR, orchestration, and the code you must read — 34%

• Ingestion Patterns — The first decision in any pipeline: streaming or batch, and what each commits you to. Throughput and latency targets, bounded versus unbounded data, replayability, the cost of getting it wrong in both directions, and the AWS service that matches each shape. (4 lessons, 55 min)
• Streaming at Scale — Kinesis Data Streams, Managed Service for Apache Kafka, and Firehose as three different bargains: who owns the shard or partition, how ordering is guaranteed and at what granularity, what happens when a consumer falls behind, and when a fully managed delivery stream is the right answer instead. (5 lessons, 65 min)
• Transformation Engines — Glue, EMR, and the Spark underneath both: where a Glue job beats a cluster and where it does not, DPUs and executors as the same idea priced differently, the file-size and partition problems that dominate real pipeline cost, and why a slow job is usually a shuffle rather than a CPU. (5 lessons, 70 min)
• Orchestrating the Pipeline — Turning a set of jobs into a dependable line: Step Functions, Managed Workflows for Apache Airflow, EventBridge and Glue workflows compared on the axes that actually decide between them — dependency complexity, failure semantics, retry and idempotency, cost at idle, and who maintains the DAG. (5 lessons, 60 min)
• Programming Concepts on the Exam — Task statement 1.4, which readers under-prepare because it looks like a coding question and is not. SQL that the exam expects you to read, infrastructure as code, Git flow and CI/CD for pipelines, the SDK patterns that appear in stems, and optimising a query by reading its plan. (4 lessons, 55 min)