Class imbalance and bias mitigation
Lesson 4 of 5 in Data Quality, Labeling, and Bias.
When 0.5% of transactions are fraud, a model that answers "legit" every time scores 99.5% accuracy and catches nothing. That is class imbalance in one sentence: accuracy becomes a lie, and the real conversation moves to precision, recall, F1, and PR-AUC — plus a decision threshold set by the business cost of each error. Fixing the data side has a standard toolkit, every item of which the exam has tested.
Resampling
Undersample the majority — fast, but throws information away. Oversample the minority — keeps everything, but duplicated rows invite overfitting. Both are train-set-only operations.
SMOTE-style synthesis
SMOTE interpolates new minority points between existing neighbors in feature space, instead of copying rows. Richer than naive oversampling — and like it, applied after the split, to training data only.
Class weights
Cost-sensitive learning: make minority mistakes expensive in the loss (e.g. XGBoost's scale_pos_weight). No data surgery at all — often the first thing to try, since the dataset stays honest.
Metrics & thresholds
Judge with PR curves, F1, or recall at a required precision — and remember the decision threshold is a free parameter: moving it trades precision for recall without retraining anything.
Grouped horizontal bar chart with four bars in two pairs. Before rebalancing, the majority class holds 97 percent of training rows and the minority class 3 percent. After rebalancing the training split, each class contributes roughly 50 percent of the training signal. Evaluation sets keep the original imbalanced ratio.
Imbalance is one source of bias; the exam wants the fuller census. Sampling bias — who got into the dataset (one region's customers, one hospital's patients). Measurement bias — the proxy differs in quality across groups (camera quality, translated text). Label/historical bias — the labels encode past human decisions, discrimination included. The data-layer countermeasures are the ones you already know — stratified splits, reweighting or resampling per group, targeted collection and augmentation for underrepresented groups — applied with a fairness lens, and the same logic extends conceptually to multimodal datasets (an image corpus skewed by demographic needs balanced collection or augmentation just as a tabular one does).
On AWS the pre-training measurement tool is SageMaker Clarify: it computes bias metrics on the data itself — class imbalance (CI) and difference in proportions of labels (DPL) across a chosen facet such as age or gender — surfaced directly in Data Wrangler's bias report, before any model exists to blame.
As of October 2026 Clarify is closed to new customers (existing customers continue; AWS maintains it but plans no new features), exactly as Model Monitor is — the two went into maintenance together, because Model Monitor's bias-drift and feature-attribution monitors are Clarify running on a schedule. MLA-C02 still names and tests it, so the metrics above are live exam content. What the lifecycle change affects is only the "what would you build today" answer; the measurement concepts are the transferable part.
Key terms: SMOTE, SageMaker Clarify, Data leakage
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.