Masking, redaction, and tokenisation

Lesson 4 of 5 in Encryption and Masking.

Encryption protects data from someone who should not have it. Masking protects data from someone who should — the analyst who needs the table but not the card number, the support agent who needs the account but not the national identifier. That is a different problem with a different answer, and the exam's actual question is never "which one looks most protected". It is what does each treatment still let you do.

Five rungs, and the useful way to hold them is by three properties: is the output deterministic (equal inputs give equal outputs), does it preserve ordering, and is it reversible by someone authorised?

• Full redaction — NULL or a constant. Nothing survives. Note the alternative that is often better: do not grant the column. A nulled column is present and breaks nothing; an ungranted column makes SELECT * fail, which tells the reader the truth.
• Partial redaction — ****8842. Deterministic, so it looks joinable, and it is not: two cards sharing a suffix are different cards, so a join on it is wrong rather than merely absent. The revealed fragment is genuine data, and where the hidden part is derivable — an identifier whose leading digits encode a birth date and region — it protects very little.
• Tokenisation — a surrogate from a vault, often format-preserving so schemas do not change. Deterministic, so joins and COUNT(DISTINCT) are exactly correct; reversible through a vault lookup for the few principals that may. This keeps the most analytical capability while removing the value from the analytics estate, and the price is that the vault becomes a thing to protect, audit and back up.
• Salted hash — joins like a token with no vault to operate. The caveat is sharp: "irreversible" is a claim about entropy, not about the algorithm. An unsalted digest of a card number, a national identifier or an email address is recovered by hashing the candidate space, which for most identifier formats is small enough to exhaust. A secret salt or a keyed HMAC fixes that and reintroduces a secret every joining system needs.
• Encryption of the field — reversible with the key, and not deterministic under standard modes, so it destroys joins. Right when the value must travel encrypted and be decrypted by one consumer; wrong when analysts must still group by it.

Ordering is lost by all of them, which is a design constraint people discover late: a tokenised amount cannot answer "above a threshold". If a range predicate is needed, keep the real value in a protected column or derive a separate band column at ingest.

And the distinction that decides whether an obligation still applies: pseudonymisation versus anonymisation. Tokenised and salted-hash data can still be re-linked given extra information, so it remains personal data — deletion requests still reach it, residency rules still bind it. Genuine anonymisation means the link cannot reasonably be re-established, which takes more than dropping the name column, because unique combinations of postcode, birth date and job title identify people perfectly well.

Where you apply it matters as much as which one you pick, and the exam phrases this as a preposition: masked at the destination or masked at the source.

At the destination means the data lands in full and is masked on the way out. Redshift dynamic data masking is the clearest example: the base table holds the real values and the policy rewrites them per role at query time. The appeal is that it needs no pipeline change and existing dashboards keep working. The limitation is exactly as large: a superuser sees the originals, an UNLOAD to Amazon S3 writes them in the clear because no policy is in that path, and a backup contains them. Destination masking controls reading, not storage.

At the source means the sensitive value never reaches the store. AWS Glue sensitive data detection runs inside the ETL job: it matches managed entity types and custom patterns, then takes an action in the same job — report the finding, redact, partially mask (with offsets or a regex), or replace with a hash. Fine-grained action overrides let you set a different action per column and entity type, and exclude columns you know are clean so the scan does not pay for them. Two documented details worth holding: the two scan modes are find sensitive data in each row (comprehensive, and the only mode that supports fine-grained actions) versus find columns that contain sensitive data (samples, cheaper, locates rather than fixes); and two different edit-in-place actions — redaction and hashing — cannot both apply to the same column, though detection always can.

The rule that follows: if the requirement says the sensitive value must not be written to the destination, destination masking is the wrong answer. Transform at the source, or in a Lambda transformation on a Firehose stream before delivery. If the requirement says existing consumers must keep working while values stop being readable, destination masking is right.

A genuinely common production shape, worth recognising as an answer: tokenise at ingest, keep the vault in a separate account. The lake and the warehouse then contain no card numbers at all, analysts keep every join and count they had, a deletion request is satisfied by removing one vault entry — which severs the link everywhere at once — and the small set of people who may detokenise do so through an audited service. It costs you range predicates and a vault to operate, and it is usually the right trade.

Three other masking surfaces appear in questions. AWS Glue DataBrew has masking, hashing and substitution transforms for preparation work done visually. CloudWatch Logs data protection policies mask matched values in log events, with logs:Unmask as the separate permission that reveals them — the control for "engineers may read the log but not the card numbers in it". And Amazon Comprehend detects and redacts personal data in free text, which is where unstructured fields that no column-level control can reach are handled.

Key terms: Tokenisation, Irreversible hash (with salt), Partial redaction, Pseudonymisation vs anonymisation, AWS Glue sensitive data detection, Data masking

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.