Feature engineering: scaling, encoding, binning
Lesson 2 of 5 in Transforming Data and Engineering Features.
Models eat numbers, and they are picky about the menu. Scaling matters wherever distance or gradients are involved: k-NN and SVMs compare feature distances (a salary in the hundred-thousands will drown an age in the tens), and linear models and neural networks converge faster when features share a scale. Tree ensembles like XGBoost split on thresholds and genuinely do not care — a fact the exam uses as a distractor in both directions.
Standardize (z-score)
Subtract the mean, divide by the standard deviation → mean 0, variance 1. The default for linear models, SVMs, and neural networks. Outliers still pull the mean and variance, so extreme values keep some influence.
Min-max normalize
Rescale into [0, 1]. Preserves the shape of the distribution and gives hard bounds — useful when the algorithm expects bounded inputs. Fragile to outliers: one extreme value compresses everyone else into a sliver of the range.
Robust scale
Center on the median, scale by the IQR. Because both statistics ignore the tails, this is the scaler to name when the stem says "contains significant outliers" but you must keep them.
Log transform
Compresses right-skewed, long-tailed positives (income, counts, transaction amounts) toward symmetric; use log1p when zeros exist. Turns multiplicative relationships into additive ones that linear models can learn.
Categorical features get encoded, and the fork is tested constantly. One-hot encoding gives each category its own 0/1 column — correct for nominal data (city, payment method) because it invents no order. Label/ordinal encoding maps categories to integers — correct only when the order is real (S < M < L); applied to nominal data it fabricates distances ("Chicago is twice Boston") that mislead linear models, though tree models largely forgive it. High cardinality (user IDs, ZIP codes) breaks one-hot with thousands of near-empty columns — reach for hashing, target encoding, or learned embeddings instead.
Binning discretizes a numeric column: equal-width bins are simple but collapse under skew (most rows pile into one bin); quantile bins put equal counts in each bin, surviving skew and capping outlier influence.
Interactive sorting exercise: Match the symptom to the technique.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.