Multimodal pipelines: text, image, audio, tabular
Lesson 2 of 3 in Data Pipelines for FM Consumption.
Production inputs rarely arrive as clean text. Four shapes converge on one context window, and each needs its own preparation: text (normalize, clean, de-boilerplate), images (feed Bedrock multimodal models natively, or caption at ingest when the index is text-only), audio (Amazon Transcribe turns speech into text first — the example the AIP-C01 exam guide itself uses), and tabular data (serialize rows schema-aware into text or JSON the model can actually read). The heavy lifting at corpus scale runs in SageMaker Processing jobs; the per-request assembly is typically Lambda.
| Data type | The move | Watch out for |
|---|---|---|
Text | Clean, normalize, strip boilerplate — Lambda or Bedrock itself reformatting text | Encoding corruption and extraction junk embed as confidently as prose |
Image | Native image blocks for multimodal FMs; caption at ingest for text-only indexes | Model input limits on image size/count; captions flatten layout nuance |
Audio | Amazon Transcribe first (custom vocabulary for domain terms), then the text path | Transcription errors propagate downstream — gate transcript confidence |
Tabular | Schema-aware serialization via SageMaker Processing / Data Wrangler exports | Dumping raw CSV: embeds poorly, wastes tokens, loses column semantics |
Two pre-trained AI services complete the on-ramp picture. Paper: Amazon Textract turns scanned documents and PDFs into text, tables, and form key-value pairs — the extraction stage that runs before chunking and embedding whenever the corpus is scans rather than born-digital text. Pixels: Amazon Rekognition reads images themselves — labels, faces, text-in-image, and content moderation — which is also the safety gate of a multimodal pipeline: moderate user-supplied images before they join an FM request, because a guardrail policy that inspects only prompt text never sees pixels.
Interactive checkpoint quiz (4 questions) — open this page in a browser to take it.