Retention, immutability, and protecting the evidence
Lesson 4 of 4 in The Logging Architecture.
Ask one question of every log archive you design: who could destroy this evidence?
Enumerate honestly. The workload account’s administrators. A compromised CI/CD role with broad S3 permissions. The organization’s management-account root user. An attacker who got far enough to notice that their activity is being recorded — and a competent attacker’s second action, after establishing access, is to stop the logging and delete what it already wrote. If the answer to “who could destroy this?” includes anyone who could plausibly be the subject of the investigation, the archive is decoration.
The answer is built in layers, and the layers are cumulative — each one assumes the ones beneath it. Read the ladder upward.
Tamper resistance, bottom-up. Each layer removes a different destroyer.
- Versioning — a delete is now a delete marker
With versioning on,
DeleteObjectwithout a version id adds a delete marker and the data remains as a non-current version. This is the foundation every layer above relies on — Object Lock requires versioning, and MFA delete is a versioning setting. It also introduces the failure it is famous for: a lifecycle rule expiring non-current versions quietly undoes the protection, so audit your lifecycle rules against your retention promise, not just against your bill. - Separate account ownership — the evidence is not where the incident is
The log bucket lives in a dedicated security account that the workload teams have no administrative access to, and delivery arrives cross-account with
bucket-owner-full-controlso the receiving account owns every object. This one architectural choice removes the largest class of destroyer at a stroke: compromising a workload account no longer grants access to that workload’s own evidence. It is also why organization trails and central logging accounts appear in nearly every correct answer in this domain. - Explicit deny, including for admins — no principal has a delete path
An explicit
Denyin the bucket policy ons3:DeleteObject,s3:DeleteObjectVersion,s3:PutBucketPolicyands3:PutLifecycleConfiguration— with the narrowest possible exception, or none at all — beats anyAllow, including an administrator’s. Reinforce it from above with a service control policy so the member account cannot edit the bucket policy back. The mental test: if a single named human can remove the evidence, the control is a convention, not a control. - MFA delete — a physical factor on the destructive path
Enabling MFA delete on a versioned bucket means permanently deleting an object version, or suspending versioning, requires an MFA code in the request. Two properties the exam tests: it can only be enabled by the bucket owner’s root user using the CLI or API — not the console, not an IAM user — and it protects version deletion and versioning state specifically. Operationally awkward by design, which is the point for an archive nobody should be editing.
- Object Lock, compliance mode — nobody, including root, until the date
S3 Object Lock in governance mode blocks deletion but allows a principal holding
s3:BypassGovernanceRetentionto override — appropriate when you want friction, not finality. Compliance mode permits no override by anyone, root included, until retention expires, and a retention period cannot be shortened once set. A legal hold is an independent open-ended flag with its own permission. For the older Glacier vault API the equivalent is Vault Lock: initiate, test for 24 hours, then complete the lock and the policy is permanent. This is the top of the ladder because it is the only rung that answers “who could destroy this?” with nobody.
Retention itself is a security requirement with two edges, and the exam phrases scenarios to make you choose between them.
Too short and the evidence expires before anyone asks. Industry breach-detection timelines run to months; a 30-day retention against an incident discovered on day 120 is an unanswerable question, and “we deleted it on schedule” is not a defence anybody accepts. Compliance frameworks frequently set floors — one year readily available, often longer in archive.
Too long and you are holding data you promised to delete. Logs contain IP addresses, user identifiers and sometimes content; an indefinite archive can itself breach a privacy commitment and is certainly an expanded breach surface. “Never expire”, which is the default on a hand-created CloudWatch log group, is a decision, not an absence of one.
The resolution is tiering rather than a single number: a hot window that is cheap to query, a warm archive that is cheap to store, and a cold archive that is cheap to keep but slow to retrieve. S3 Lifecycle rules do the moving. Match the tier boundaries to how investigations actually run — days for triage, months for scoping, years for audit — and remember that a lifecycle rule which expires objects is a destroyer too: Object Lock will refuse the deletion, which is exactly the argument for having it.
| Window | Where it lives | Why | Protection applied |
|---|---|---|---|
0–30 days (triage) | CloudWatch Logs, and/or an OpenSearch hot index | interactive querying during live incidents; alarms and metric filters need it here | log group KMS encryption; retention set explicitly, never left at never-expire |
0–12 months (scoping) | S3 Standard / Standard-IA in the security account, Parquet where possible | Athena queries over months of history at a sane cost | versioning, explicit deny on delete, Object Lock governance or compliance mode |
1–7 years (audit, legal hold) | S3 Glacier storage classes via a lifecycle transition | regulatory retention at archive pricing; retrieval is rare and can be slow | Object Lock compliance mode for the stated term, or Glacier Vault Lock on legacy vaults |
Integrity, throughout | CloudTrail digest files in their own prefix | proving the archive was not altered is a separate requirement from preventing alteration | log file integrity validation enabled on the trail; tighter permissions on the digest prefix |
One more control deserves naming because it is elegant and the exam likes it: a KMS key policy that lets the producer write but not read.
Grant the log-producing service the operations it genuinely needs — kms:GenerateDataKey and kms:Encrypt for the service principal, scoped with a source-ARN condition — and grant kms:Decrypt only to the security account’s analyst roles. The workload account can then deliver logs it cannot subsequently read, which breaks a real attack path: a compromised application role that could read the archive could learn exactly what evidence exists about it, and selectively target the gaps. Writing without reading is the asymmetry you want, and it is enforced in the key policy rather than in anyone’s good intentions.
Combine it with the rest and the archive has a single, boring property: logs go in, nothing comes out except to a small set of named roles in an account that the subject of any investigation cannot administer. That is the design the exam is reaching for whenever a scenario mentions an insider, a compromised administrator, or an auditor who wants assurances.
Key terms: S3 Object Lock, S3 Glacier Vault Lock, Log file integrity validation, Organization trail
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.