Transformation Engines

Glue, EMR, and the Spark underneath both: where a Glue job beats a cluster and where it does not, DPUs and executors as the same idea priced differently, the file-size and partition problems that dominate real pipeline cost, and why a slow job is usually a shuffle rather than a CPU.

Content current as of 2026-10.

Lessons

  1. AWS Glue ETL and its job model
  2. Spark mechanics a data engineer must know
  3. EMR: when the cluster earns its keep
  4. File formats, compression, and partitioning
  5. Diagnosing a slow transformation

Sources