The Decision Drivers

Lesson 3 of 3 in Managed APIs vs Self-Hosting.

Four drivers decide the stop. Two of them are usually misread, so take those first.

Driver 1 — data gravity and privacy. The instinct says sensitive data → self-host. The documentation says otherwise: all three clouds publish private paths to their managed model serving. Bedrock documents interface VPC endpoints via AWS PrivateLink so traffic avoids the public internet — including for model customization jobs and batch inference. Azure AI Foundry resources support disabling public network access entirely behind an Azure Private Link Private endpoint, with DNS remapped so clients keep the same connection string. Google documents Private Service Connect endpoints for reaching the generative APIs privately (including from on-premises over HA VPN or Cloud Interconnect), plus dedicated PSC-based private endpoints for self-deployed models. What privacy actually decides is residency mechanics: whether you can use Cross-region inference and global routing (which trades residency control for availability) or must pin processing to a geography — a choice every platform documents explicitly, with geographic profiles, Data Zone deployments, and regional endpoints as the residency-preserving options. And data gravity cuts one more way: Egress charges and sheer dataset mass pull the model toward the cloud where the data already lives.

Driver 2 — control. If a catalog model as served meets the requirement, stops 1–2 win by default. You move right only for things the managed API cannot express: weights the catalogs do not carry, a fine-tune of an unsupported architecture, a pinned Serving engine version, custom Speculative decoding or Quantization choices. Note how much control the platforms have absorbed before you get to stop 4: Model Garden documents self-deployment with custom containers such as vLLM, Bedrock Custom Model Import serves imported Open weights models back on on-demand throughput, and Foundry fine-tunes non-OpenAI models on managed compute (using your own VM quota). The list of things that truly require your own cluster is shorter every year.

Driver 3 — cost shape. Do not decide from list prices in prose — decide from shapes, then run your own numbers in the tools. Pay-per-token is a pure variable cost: it scales to zero and it scales to painful. Committed capacity is a step function: a fixed spend per term that beats the variable cost only above a break-even volume. Dedicated compute (stops 3–4) is a fixed cost with Utilization risk: the instance bills whether it serves or idles.

The break-even is a formula, not folklore. For sustained monthly volumes V_in and V_out (tokens) at on-demand rates P_in and P_out (per token), against a commitment of N capacity units at rate R per unit-hour over H hours: the commitment wins when V_in × P_in + V_out × P_out > N × R × H. For dedicated compute, the same logic runs through utilization u: capacity billed for H hours but busy for u × H effectively costs R / u per useful hour, so halving utilization doubles your real unit cost. Every variable is measurable — plug your traffic into the token cost estimator and the GPU sizing tool, and go deeper in Cost Modeling, where commitments, caching, and spillover get the full treatment.

Driver 4 — team maturity. The honest question is not can we self-host — a weekend proves you can — but can we still run it in month eighteen: engine CVEs patched, driver upgrades rehearsed, autoscaling tuned, capacity reservations or spot strategies managed, on-call staffed. Each stop rightward converts provider headcount into your headcount. If that trade is not deliberate, the spectrum has a default direction: start left, move right only when a driver forces it.

Key terms: Provisioned throughput, Managed endpoint, Model catalog, Quota, Private endpoint, Utilization

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.