Event-driven remediation

Lesson 1 of 3 in Automated Response and Forensics.

Every automated-response question on this exam is a variation of the same three-station spine: something is detected → an EventBridge rule matches it → a target does the work. Learn the spine and the questions collapse into one decision — which target.

The sources on the left are familiar from detection: GuardDuty findings, Security Hub findings (both new imports and workflow-status changes), AWS Config compliance changes, CloudWatch alarm state changes, Inspector findings, Macie findings, Health events, and anything else that lands on the default event bus — including CloudTrail API calls themselves. The rule in the middle is an event pattern, and its precision is the difference between automation you trust and automation you disable after a week: match on finding type and severity and account and resource tags, not on "GuardDuty said something".

The target on the right is where the design judgement lives, and the four that matter are Lambda, Systems Manager Automation, Step Functions, and SNS.

Now the target choice, properly. The exam does not ask "can Lambda do this?" — Lambda can do almost anything, which is why it is the default wrong answer. It asks which target is designed for the job, and the discriminators are specific.

Systems Manager Automation wins whenever the action happens on an instance. The document runs through the SSM Agent, so the responder needs no SSH, no bastion, and no inbound rule — which matters enormously when the instance you need to inspect is the one you just cut off the network. Automation documents also carry approval steps, branching (aws:branch), rate control across many targets, and their own IAM assume-role, and because they are documents rather than code they are the natural home for a runbook written by the team that owns the resource.

Step Functions wins whenever the response is a workflow: several steps, a decision between them, a wait, a retry policy per step, or a human approval. Its other property matters in forensics — it keeps execution state durably and shows you the exact path taken, which is both the audit trail and the debugging surface. Lambda wins when the response is pure API work with a little logic and must happen in seconds. And SNS is not a remediation at all: it is how humans learn, and an answer offering SNS alone to a question that says "automatically remediate" has stopped one station short.

Choosing the EventBridge target — the discriminators the exam actually uses
Target	Reach for it when	Its distinguishing property	Where it goes wrong
Lambda	The response is a handful of API calls and some logic, needed in seconds	Arbitrary code, fastest to write, 15-minute maximum runtime	Multi-step responses become untestable code; long waits and approvals do not fit
SSM Automation	The action happens on an instance — collect, isolate, patch, reboot	Reaches managed instances through the SSM Agent, with no inbound network path or SSH key	Requires a managed instance with the agent running and an instance profile; useless on an unmanaged host
Step Functions	The response has branches, waits, per-step retries, or a human approval	Durable execution state and a visible path through the workflow — audit trail by default	Overkill for a single API call; adds a state machine to maintain
SNS	Humans need to know — on its own, or as a step inside the others	Fan-out to people and systems, including into a chat channel	It is notification, not remediation. Offered alone against "automatically remediate", it is the trap answer
Incident Manager response plan	The event should open an incident and engage on-call with escalation	Creates the incident record, pages contacts, and can start runbooks together	Not a remediation path either — it mobilises people rather than changing a resource

Two more pieces complete the automation picture. The first is Amazon Application Recovery Controller, which the exam guide names alongside the compute targets and which does something none of them do: it orchestrates recovery rather than remediation. Its three capabilities map cleanly onto incident work. Readiness checks continuously audit whether your standby — the other Region, the other cell — is actually configured and scaled to take the traffic, so "we can fail over" is a measured claim rather than a hope. Routing controls are highly available on/off switches, backed by a cluster with a data plane deliberately independent of the Region you might be evacuating, with safety rules that stop you turning everything off at once; you flip a routing control and Route 53 health checks follow it. And zonal shift moves traffic away from an impaired Availability Zone with a single call. When a question describes shifting traffic away from a compromised or impaired cell as part of the response, hand-editing DNS records is the wrong answer and a pre-built routing control is the right one.

The second is the discipline that makes any of this safe to run unattended, and it is the part teams learn the hard way.

The safety gate every auto-remediation needs before it is allowed to act

Matched finding arrives
The event pattern already narrowed by type, severity, account and tags.
Is the action reversible?
Replacing a security group is reversible. Terminating an instance, deleting a key, or disabling a user in production is not — those belong behind a human approval.
Is the resource excluded by tag or OU?
An explicit exclusion list — a tag like ir:no-auto-remediate, or an OU the automation never touches — is how a team keeps one fragile legacy workload out of harm’s way without disabling the automation entirely.
Has this already fired for this resource in the window?
The circuit breaker. Count actions per resource and per window in a DynamoDB table or a CloudWatch metric; the same finding re-delivered, or a detector stuck in a loop, must not produce repeated action.
Act idempotently, then record
Make the action safe to repeat — set the security group to the known isolation group rather than "remove the current rules" — then write the outcome to the finding and the incident record.
Notify, open an OpsItem, wait for a human
Step Functions holds a wait-for-callback task; SNS notifies; the approval returns with a token. Nothing destructive happens before it does.
Suppress and alarm on the loop
Stop acting and raise an alarm that a human must clear. A silent suppression is just a broken automation.
Finding updated, action logged
Workflow status moved, automation execution id recorded, metric emitted. The next engineer can see what happened and why.

Automation that amplifies the incident

The failure modes are all real and all cheap to prevent. Double delivery: EventBridge delivers at least once, so a non-idempotent remediation can run twice — write actions that converge on a desired state rather than applying a delta. The loop: your remediation changes a resource, the change produces a Config or CloudTrail event, the event matches your rule, and you have built a machine that fights itself. The production outage: an automation that revokes a security group rule, matched too broadly, taking the payments service offline faster than any attacker would have. The silent failure: no dead-letter queue on the rule, no alarm on target failures, so the automation stopped working in March and nobody noticed until October.

Guardrails, in order of value: a precise event pattern, an exclusion tag honoured by every automation, idempotent actions, a per-window circuit breaker, a dead-letter queue, an alarm on invocation failures — and a human approval in front of anything irreversible.

Key terms: Systems Manager Automation document (runbook), Amazon Application Recovery Controller (ARC), Remediation idempotency, Amazon EventBridge

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.