Metadata frameworks, sharding, hierarchical indexes
Lesson 2 of 3 in Vector Stores at Professional Scale.
Vector-store design hides two different levers, and professional stems test whether you can tell them apart. Metadata frameworks buy precision and context awareness: S3 object metadata carrying document timestamps, custom attributes carrying authorship, tagging systems carrying domain classification — stored beside each chunk so retrieval can filter ("only current-year policies", "only legal-team documents") instead of hoping similarity alone lands in the right subset.
Index architecture buys performance at scale: sharding splits one large OpenSearch index across nodes so queries fan out in parallel; multi-index approaches give specialized domains their own indexes with a router choosing per query; hierarchical indexing searches coarse first (summaries, sections) and descends to fine (chunks) only where the coarse layer pointed. Precision problems get metadata; latency-at-scale problems get architecture.
Interactive sorting exercise: Precision or scale? Sort each measure by the problem it actually solves: metadata framework (search precision, context awareness) or index architecture (performance at scale).
The scale ladder, top down
- Multi-index routing — which corpus?
Specialized domains get their own indexes; a router (metadata, classifier, or agent) picks the index per query. Each index stays small, fast, and independently tunable — and a noisy corpus cannot pollute its neighbors.
- Sharding — spread one index
Within one index, shards distribute vectors across nodes; queries fan out in parallel and merge. The OpenSearch lever for keeping P99 flat while the corpus grows by an order of magnitude.
- ANN structure — inside each shard
Each shard answers with an approximate-nearest-neighbor structure — typically HNSW graphs (also IVFFlat in pgvector): logarithmic-ish search instead of scanning every vector.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.