Labeling and annotation pipelines
Lesson 2 of 5 in Data Quality, Labeling, and Bias.
Supervised learning starts with an unglamorous dependency: somebody has to say what the right answer was. SageMaker Ground Truth is the labeling pipeline — you point a labeling job at objects in S3, pick a task UI (bounding boxes, text classification, custom), and choose a workforce: the public crowd (Amazon Mechanical Turk), a vendor workforce from AWS Marketplace, or a private workforce of your own people. Quality comes from redundancy: several workers label each item and Ground Truth consolidates their answers into one label with a confidence.
The cost lever is automated data labeling: Ground Truth trains a model on the labels collected so far, lets it auto-label the items it is confident about, and routes only the uncertain remainder to humans — active learning in production clothing, and the reason million-object jobs are affordable.
A Ground Truth labeling job
- Unlabeled objects in S3
An input manifest lists the objects to label.
- Labeling job
Task UI + instructions + workforce choice.
- Auto-label confident?
With automated data labeling on, a model trained on labels-so-far predicts first.
- Human workforce labels
Public (Mechanical Turk), vendor, or private — several workers per item.
- Consolidation
Worker answers merge into one label with a confidence score.
- Labeled manifest in S3
An augmented manifest ready to feed training.
Two distinctions the exam leans on. First, labeling vs review: Ground Truth creates training data before training; Amazon A2I (Augmented AI) inserts human review of model predictions in production — low-confidence inferences get a second, human look. Same humans, different end of the pipeline. Second, the workforce is a privacy decision: medical images, legal contracts, anything confidential → private workforce, never the public crowd.
Key terms: SageMaker Ground Truth
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.