Ensuring Data Quality

The task statement with the least obvious service answer, because quality is a design problem first. The failure modes that reach production, completeness and consistency and freshness as measurable things, Glue Data Quality and Deequ, where a quality gate belongs in the line, and what to do when bad data has already landed.

Content current as of 2026-10.

Lessons

  1. How data goes wrong in production
  2. Making quality measurable
  3. Glue Data Quality and Deequ
  4. Gates, quarantine, and backfill

Sources