Evidence first: capture before you change
Lesson 2 of 5 in Responding to Security Events.
One rule governs this lesson and it is worth saying before anything else: capture before you change. Every containment action alters the scene, and some containment actions delete the scene. The sequence that keeps both properties — the attacker stopped, the evidence intact — is isolate, capture, then change, and the reason isolation comes first is that isolation is the one containment action that destroys nothing.
The constraint behind the rule is the order of volatility: different evidence survives for different lengths of time, so you collect in order of how fast it disappears. On AWS that order is unusually crisp, because the platform tells you exactly what dies when. Memory dies when the instance stops, reboots, or terminates. Instance-store volumes die when it stops or terminates. EBS volumes survive a stop and die on terminate if delete-on-termination is set. Logs still sitting on the host die with the volume. Logs already shipped to CloudWatch Logs or S3 survive the host entirely. And artefacts in an Object Lock bucket survive even a responder with a typo and an attacker with your credentials.
A six-layer stack showing how long different evidence classes survive on AWS, read bottom-up in capture order. Layer one, emphasized: CPU and memory state — running processes, injected code, decrypted secrets, open sockets and caches — destroyed by any stop, reboot, or terminate, so it is captured first or not at all. Layer two: instance store volumes, the ephemeral NVMe or SSD disks, whose contents are lost on stop and on terminate. Layer three: EBS volumes, which survive a stop but are deleted on terminate when delete-on-termination is set, and which are captured by snapshot while the instance runs. Layer four: logs that exist only on the host and have not yet been shipped, which rotate, get overwritten, or die with the volume. Layer five: logs already delivered to CloudWatch Logs, S3, or an organization CloudTrail trail, which survive the instance entirely and are therefore copied rather than rescued. Layer six: the immutable archive — an S3 bucket with Object Lock or a vault with Vault Lock — which survives both a mistaken responder and an attacker holding your credentials, and is where every captured artefact ends up.
Collection is only half of it. An artefact nobody can vouch for is data, not evidence, and the difference is the chain of custody: an unbroken record of what was collected, when, by whom, from where, and everything that has happened to it since. Four habits produce it, and all four are cheap at the time and impossible to retrofit.
Write to a store nobody can rewrite. The evidence bucket lives in the forensics account with Object Lock enabled, so artefacts are write-once for their retention period. Versioning on, deletion denied by policy to everyone including responders, and a bucket policy that only the forensics role can read.
Hash everything, immediately. A checksum recorded at capture time is what lets you prove months later that the image you are analysing is the image you took. The platform helps: snapshot ids are immutable references, S3 stores checksums, and CloudTrail records the capture API calls themselves with their own timestamps.
Record time unambiguously. Everything in UTC, with the source of the clock noted, because the one thing every incident report must survive is a reader in a different timezone six months later. AWS makes this easier than most environments — CloudTrail, findings, and flow logs are all timestamped in UTC — so the discipline is simply never to convert, and never to write a time without its zone.
Attribute every action. Responders work through the named incident-response role, with their own identity in the session, so CloudTrail answers "who took this snapshot?" without anyone having to remember. That is also why shared credentials are disqualifying in forensic work: an artefact collected by "the ops account" has no custodian.
| Artefact | Where it lives | How you capture it | What destroys it |
|---|---|---|---|
Memory | The running instance only | Acquisition via SSM Run Command, writing the image off-host | Stop, reboot, terminate — and the passage of time as the attacker’s process exits |
Instance store data | Ephemeral local disks | Copy it off the running instance; there is no snapshot | Stop and terminate |
EBS volume | The volume, independent of instance power state |
| Terminate with delete-on-termination; volume deletion |
Host-local logs | The instance’s file system | Inside the disk snapshot, or via SSM before the host changes | Log rotation, disk fill, volume loss |
Container / ECS task | A task that may already be gone | The image digest, the task definition revision, and the already-shipped log stream | Task replacement — which is usually automatic and fast |
Lambda invocation | Nowhere you can log into | The function version and code hash, its configuration, and the CloudWatch log stream | Function update or deletion — capture the version, not "the function" |
Network activity | VPC Flow Logs, Route 53 Resolver query logs, load balancer access logs | Copy the relevant window, with the query used, into the evidence store | Retention expiry — the quiet one that catches slow investigations |
API activity | CloudTrail (management always; data events only if enabled) | Export the query result and its parameters; cite request ids | Retention expiry; and data events never enabled, which cannot be fixed retroactively |
Interactive sorting exercise: An instance is confirmed compromised and still running. Sort each candidate action by whether it preserves the evidence you will need, or destroys it.
Key terms: Order of volatility, Chain of custody, Forensic artefact, Memory acquisition
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.