Operations & Support

Automating the line, querying it, watching it, and proving the data is fit to use — 22%

• Automating Data Processing — Making the line run without a human in it: scheduling and event triggers, Lambda inside a data pipeline and the places it does not belong, Glue job bookmarks and incremental processing, concurrency and the quota ceilings you meet at scale, and the automation that handles its own failures. (4 lessons, 55 min)
• Analyzing Data — Getting answers out: Athena over the lake and what makes a query cheap or ruinous, Redshift Spectrum and the federated query story, materialised views and result caching, OpenSearch for search and log analytics, and QuickSight as the consumer whose needs shape everything upstream. (5 lessons, 60 min)
• Maintaining and Monitoring — Knowing the pipeline broke before the business tells you: CloudWatch metrics, logs and alarms for data services, the Glue and EMR observability surfaces, lineage and traceability, cost monitoring as an operational signal, and the troubleshooting order that finds a cause instead of a symptom. (5 lessons, 60 min)
• Ensuring Data Quality — The task statement with the least obvious service answer, because quality is a design problem first. The failure modes that reach production, completeness and consistency and freshness as measurable things, Glue Data Quality and Deequ, where a quality gate belongs in the line, and what to do when bad data has already landed. (4 lessons, 55 min)