The Spectrum

Lesson 1 of 3 in Managed APIs vs Self-Hosting.

Strip away the product names and every way of running an Large language model (LLM) in the cloud lands on one spectrum with four stops.

Stop 1 — serverless token API. You call a shared, provider-hosted model and pay per Token. From your point of view there is no infrastructure at all: no instance, no Serving engine, no capacity decision beyond a Quota.

Stop 2 — provisioned / dedicated throughput. Same API, same managed model — but you buy guaranteed capacity for a period of time instead of metering every token. The meter changes from usage to committed throughput, and Utilization of that commitment becomes your number to watch.

Stop 3 — managed compute endpoint. Now you choose the weights — an open model from the Model catalog, or your own fine-tune — and deploy them to a Managed endpoint the platform provisions and operates. You pick the instance; the meter moves from tokens to compute time.

Stop 4 — self-managed on Kubernetes. You rent GPU instances or a managed cluster and run the serving stack yourself — engine, drivers, autoscaling, patching. Maximum control, and every operational concern from Serving in Production is now on your pager.

These are not maturity levels, and moving right is not graduating. They are four contracts, and mature organizations hold several at once — chosen per workload, not per company.

The consumption spectrum: four stops, one trade

  1. One model workload

    Same prompts in, same tokens out. What changes stop to stop is the contract underneath: what you are billed for, what you control, and what you operate.

  2. Stop 1 · Serverless token API

    Meter: per token. You control: the request. You operate: nothing below the API. All three clouds now tier this meter — Bedrock documents Standard, Priority, and Flex request tiers; Vertex AI documents the same three PayGo tiers; Foundry serverless splits into standard, provisioned, and batch categories.

  3. Stop 2 · Provisioned / dedicated throughput

    Meter: capacity units × time. Bedrock sells Model Units billed hourly (no-commitment, 1-month, or 6-month terms); Azure sells PTUs; Vertex AI sells GSUs on 1-week to 1-year plans. You control: guaranteed capacity. You operate: the utilization math.

  4. Stop 3 · Managed compute endpoint

    Meter: the compute under the endpoint (Foundry documents VM core-hours for managed compute). You control: the weights and the instance. You operate: sizing and scaling policy — the platform provisions and runs the endpoint. Examples: Bedrock Marketplace models deploy to SageMaker AI-managed endpoints; Model Garden self-deploys into your project and VPC.

  5. Stop 4 · Self-managed on Kubernetes

    Meter: instance-hours of your nodes, busy or idle. You control: everything — engine, batching, quantization, drivers. You operate: everything too: upgrades, CVEs, autoscaling, on-call.

  6. Four contracts, chosen per workload

    Control and ops burden rise together — you cannot buy one without carrying the other. The decision drivers in lesson 3 tell you where to stand.

Watch three dials as you move right.

The meter. Serverless bills tokens — and the meter itself is now tiered: Bedrock documents Standard, Priority, and Flex request-level service tiers plus an account-level Reserved tier; Vertex AI documents the same Standard/Priority/Flex split for pay-as-you-go. Provisioned throughput bills capacity × time in abstract units — Model Units, PTUs, GSUs — each defined by the provider, not by GPU count. Managed compute endpoints bill for the compute under the endpoint (Foundry states VM core-hours explicitly). Self-managed bills raw instance-hours whether the GPUs are busy or idle.

The control. Serverless gives you a request surface. Provisioned adds a capacity guarantee. Managed endpoints add the weights, the container (Model Garden documents prebuilt or custom containers such as vLLM), and the instance choice. Self-managed adds the entire stack: Continuous batching configuration, Quantization choices, engine versions, kernel flags.

The ops. This dial only goes up. Each step right converts a provider guarantee into a team responsibility — the full inventory is lesson 2.

AWS

On Amazon Bedrock the stops read as: on-demand serving of shared foundation models with pay-per-token pricing (Standard, Priority, and Flex request tiers, plus an account-level Reserved tier with a term commitment); Provisioned Throughput purchased in Model Units, where each MU defines a per-minute input- and output-token processing level, billed hourly with no-commitment, 1-month, or 6-month terms; and Bedrock Marketplace models, subscribed to and deployed to an endpoint managed by SageMaker AI (with a documented bring-your-own-endpoint option). Custom Model Import is a notable shortcut back left: imported open-weights models are served with on-demand throughput through the standard invoke APIs. Stop 4 is plain EC2 or EKS with your own serving stack.

Azure

Azure AI Foundry documents two deployment options for catalog models: serverless, where Microsoft hosts the model and bills for inputs and outputs (typically in tokens) — subdivided into standard (pay-per-token), provisioned (reserved capacity), and batch categories, each in Global, Data Zone, and Regional variants — and managed compute, where the model weights run on dedicated virtual machines billed by VM core-hours. An instant-access preview even lets supported models be called by name with no deployment created at all. Stop 4 is AKS with GPU node pools.

Google Cloud

Vertex AI documents five consumption options for generative models: three pay-per-token PayGo tiers (Standard, Priority at a premium rate, Flex at a discount for latency-tolerant work), Provisioned Throughput — a fixed-cost, fixed-term subscription sold as GSUs in 1-week, 1-month, 3-month, and 1-year plans, with overage above the reservation billed as standard pay-as-you-go — and Batch for high-volume asynchronous jobs. Model Garden additionally supports self-deployed models (open, partner, and custom) that run on compute inside your own project and VPC, in contrast to the serverless MaaS offerings. Stop 4 is GKE.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.