Multimodal at the Meter

Lesson 3 of 3 in Multimodal Models.

Every capability eventually meets a meter, and multimodal inputs meet it in one of two documented ways: as tokens — the encoder’s output counted against the same per-token meter as text — or per item, a flat unit per image, per second of audio, or per request. Which one applies is not a matter of principle; it is a line on the vendor’s pricing page, and it varies by vendor, by model, and by modality. The mechanism from last lesson tells you why token-metering is natural (encoded images literally are input tokens), but naturalness is not a billing contract — read the page.

A practical corollary for this site’s own tooling: the token cost estimator works in text tokens — its comparison table is text-token scoped, so an image-heavy or audio-heavy workload needs the modality’s documented metering unit added on top before the estimate means anything.

Two meters people forget: latency and guardrails

Latency. Image tokens arrive with the prompt, so they are processed during prefill — the phase that determines time-to-first-token. A prompt carrying several encoded images can multiply the positions the model must attend over before emitting its first token, while decode speed afterwards is untouched. If your latency SLO is written in time-to-first-token, multimodal inputs attack exactly that number.

Guardrails. Safety filters, moderation layers, and policy tuning are built and evaluated per modality, and coverage documented for text does not automatically extend to images or audio — instruction-like content inside an image is a well-known evasion surface. Vendors differ in what they document here; the durable rule is to test your safety stack with the modalities you actually accept, and to treat undocumented coverage as absent.

Interactive sorting exercise: Your text-only assistant is gaining image input. Sort each claim: does it change when the workload goes multimodal, or does it stay the same?

In production

On the managed platforms, multimodal capability arrives family-by-family and model-by-model — a catalog listing a family name tells you nothing about which modalities the specific hosted variant accepts, emits, or meters. The evergreen discipline: verify per model card, never per family.

AWS

Amazon Bedrock’s model catalog documents capabilities per hosted model, and modality support routinely differs between variants and versions of the same family. Treat the individual model’s documentation page as the contract — for accepted input modalities, output modalities, and how each is metered — and re-verify it whenever you swap in a newer variant, because modality support and metering can both change across versions.

Azure

Azure AI Foundry’s model catalog spans many families, and its per-model cards and capability filters exist precisely because family names do not answer modality questions. Filter by the task or modality you need, then open the specific model’s entry for its documented inputs, outputs, and deployment constraints — and run your content-safety configuration against each modality you enable, since filter coverage is configured and evaluated per modality.

Google Cloud

Vertex AI’s Model Garden lists first-party, open, and partner models with per-model documentation of supported inputs and outputs. The navigation habit is identical: the specific model’s page — not the family banner — states what a deployment accepts and how usage is counted, and video, audio, and image handling each come with their own documented limits that change from model to model.

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.