Security Lake and OCSF
Lesson 1 of 3 in Log Analysis at Scale.
Here is the problem Amazon Security Lake exists to solve, stated as a query. “Show me everything involving this IP address in the last 90 days.”
Without a common schema you write that query five times. CloudTrail calls it sourceIPAddress. A VPC flow log calls it srcaddr — in a space-separated line with no field names at all. An ELB access log has its own positional format. A third-party firewall has whatever its vendor chose. A Security Hub finding nests it several levels deep. Five parsers, five sets of assumptions, five places for the correlation to be subtly wrong — and a new log source means a sixth.
The Open Cybersecurity Schema Framework is the fix, and it is the piece SCS-C03 newly emphasises. OCSF is an open, vendor-neutral schema for security events, co-founded by AWS with industry partners: defined event classes, versioning rules for schema evolution, and a governance process. Its payoff is unglamorous and enormous — a source address has one field name regardless of which product emitted the event, so correlation stops being a parsing project and becomes a join.
Security Lake is AWS’s implementation of that idea as managed infrastructure: it collects, converts everything to OCSF in Apache Parquet, stores it in your own account, and shares it with subscribers.
Say plainly what Security Lake is not, because a well-written distractor will offer it as something it isn’t.
It is not a detector. It raises no findings of its own. It consumes GuardDuty and Security Hub findings as sources; it does not produce them. A scenario asking “which service detects the credential theft” is never answered with Security Lake.
It is not a SIEM. It has no correlation rules, no case management, no analyst workflow. It is the normalized, queryable substrate a SIEM or an analyst sits on top of — which is precisely the division of labour AWS is proposing, and the reason the subscriber model exists.
It is not free, and not instant. You pay for ingestion, the underlying S3 storage, and whatever queries you run; conversion and delivery take minutes, not seconds. For a sub-second alerting requirement, a metric filter and alarm still beats a data lake.
What it is: the answer to “centralize security logs from AWS, on-premises and third-party sources into one queryable, normalized store that we own, and let our SIEM and our analysts both read it.” When a question stacks up multi-account, multi-Region, multiple sources, third-party integration and a common schema, it is describing Security Lake and nothing else.
| Approach | Sources | Schema | Query path | Choose when |
|---|---|---|---|---|
Raw S3 archive + Athena | whatever you deliver — one table per format | none; you define tables and parse per source | Athena SQL over partitioned prefixes | you have a few formats, want the cheapest storage, and accept the table-maintenance work |
CloudTrail events, Config items, a few integrations | CloudTrail’s own event schema | SQL on a managed, immutable event data store | the question is API activity over years and you want zero pipeline | |
many AWS services, plus on-premises and third-party custom sources | OCSF — one vocabulary across every source | Athena, OpenSearch or Redshift as a query subscriber; SIEMs as data subscribers | you must correlate across many heterogeneous sources, and a SIEM must read the same corpus |
Key terms: Amazon Security Lake, Open Cybersecurity Schema Framework (OCSF), Security Lake subscriber, Apache Parquet
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.