Masking sensitive data
Lesson 2 of 4 in Secrets, Masking, and Resilience.
Masking is new in SCS-C03, and it exists because of a failure mode every mature organisation has lived through: the sensitive data was protected everywhere it was designed to live, and then it appeared somewhere nobody designed — in a log line, in a notification payload, in a support ticket pasted from a stack trace. An encrypted database does not help when the card number is in CloudWatch Logs where forty engineers have read access.
Think of it as three layers, in order of preference. Prevent it being emitted: structured logging with explicit field allow-lists, no request-body dumps, no exception handlers that serialise whole objects. Detect it when prevention fails, because it will — automated scanning that tells you a log group or a topic has started carrying personal data. Mask it so that the people with legitimate access to the log cannot read the sensitive values inside it, which separates “can read application logs” from “can read customer card numbers”. The exam names two mechanisms for the third layer and one for discovery.
CloudWatch Logs data protection policies apply at the account or log-group level. A policy names the data identifiers to look for — AWS-managed detectors for credentials, financial, personal and health data across many countries, plus custom identifiers you define with regular expressions — and an audit destination. Matching values are masked in the console and in API reads, shown as asterisks, and only a principal holding logs:Unmask sees the raw value. Two properties matter for exam purposes: the masking is applied to data as it is ingested and at read time, so it covers the log events arriving from then on; and it is a read-time control over data already in the log, which means it limits who can see the secret but does not stop the application writing it. The durable fix stays in the code. Findings go to an audit destination — CloudWatch Logs, S3, or Firehose — so you learn which log groups are carrying what.
Amazon SNS message data protection applies the same idea to messages in motion, and adds an option that logs do not have: it can refuse. A data protection policy on a topic contains statements that select data identifiers, choose a direction — inbound from publishers, or outbound to specific subscriptions — scope to accounts or principals, and then take one of three operations. Audit reports on sensitive data found, to a log group, a bucket or a Firehose stream. De-identify masks or redacts the matched values. Deny blocks the publish or the delivery outright. Direction plus operation is the combination to understand: you can allow a payload to be published but strip the personal data before it reaches an external subscriber, or you can refuse the publish entirely.
Amazon Macie is the discovery complement for S3: sensitive-data discovery jobs and automated discovery inventory your buckets, apply the same family of managed and custom data identifiers, and emit findings naming the bucket, the object and the identifier matched. Macie finds; it never modifies or redacts the object. Remediation is your pipeline’s job — which is the distinction a question will test.
| Mechanism | Where it acts | What it can do | Who can see the raw value | What it cannot do |
|---|---|---|---|---|
Account or log-group level, on log events | Audit findings and masking in the console and API reads | Only principals with | Cannot block the write — the application still emitted it; and it does not retroactively fix history written before the policy | |
Per topic, with inbound (publish) and outbound (per subscription) statements | Audit, de-identify (mask or redact), or deny the publish or delivery | Subscribers of statements that do not de-identify | Cannot scan data at rest, and does not apply to other messaging services | |
Amazon S3 — discovery jobs and automated discovery | Findings only: bucket, object, identifier matched, severity | Anyone with access to the object; Macie changes nothing | Cannot mask, redact or move anything — remediation is your pipeline’s job |
Three practical notes that turn into questions.
Masking is not deletion. A CloudWatch Logs data protection policy hides values from readers; the events still contain them, and a subscription filter or export that bypasses the console read path needs its own thinking. If a regulation requires the data not to exist, you need the logging code fixed plus a retention policy that expires the affected log group — masking alone does not satisfy “must not be stored”.
Custom identifiers are where real coverage comes from. Managed identifiers cover card numbers, national identifiers, keys and tokens, health and financial formats. They do not know your internal customer reference, your policy number format or your partner’s account scheme. A custom data identifier — a regular expression, optionally with keywords, a maximum match distance and an ignore list — is how you extend detection to the formats only you use, in both CloudWatch Logs and Macie.
Decide where masking belongs in the pipeline. Masking at the log destination protects readers of that log. Masking at the source — in the application, or with a Lambda transformation on a Firehose stream before delivery — means the sensitive value never lands in the store at all, which is strictly stronger and the right answer when a question says the data must not be written to the destination. For generative AI pipelines the equivalent control is the Bedrock Guardrails sensitive-information filter, which can mask entities in prompts and responses in flight; it is covered in the AIP line and is worth recognising as the same pattern applied to a different data path.
Key terms: CloudWatch Logs data protection policy, Amazon SNS message data protection policy, Managed and custom data identifiers, Amazon Macie, Data masking
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.