Where LLMs Live: Apps, APIs, and Weights

Lesson 3 of 4 in What Is a Large Language Model?.

Strip away the branding and an LLM is a file: billions of parameters serialized to disk. That file reaches you through exactly three doors, and knowing which door you are standing at answers most practical questions before they are asked.

Door one: a consumer app. A chat window in a browser or on a phone. Someone else chose the model, wrote the hidden instructions wrapped around your messages, runs the hardware, and updates everything on their schedule. You get convenience and zero control; you typically pay per seat or subscription.

Door two: an API. Your code sends a request to a metered endpoint and gets tokens back. The provider still runs the GPUs and serves the model, but now you choose which model, write every word of the prompt, and integrate the output into your own product. The meter counts tokens — input and output — and the bill scales with usage, including down to zero when nobody calls.

Door three: open weights. You download the parameter file itself — an Open weights release — and run it on hardware you control: your cloud GPUs, your datacenter, in the small-model case even your laptop. Nothing leaves your infrastructure, nothing changes unless you change it, and every operational concern — serving software, scaling, monitoring, updates — is now yours. You pay for compute time whether the model is busy or idle.

Same file, three doors. The differences are not about the model at all — they are about who runs the stack and what the meter counts.

A vertical stack diagram with four layers, bottom to top: model weights (the parameter file), serving stack (GPUs and inference software), API endpoint (metered and authenticated), and application (chat UI or product feature). The diagram illustrates that consumer apps, APIs, and open-weight downloads correspond to entering this same stack at different layers.

The serving stack, bottom-up. Every LLM product is this stack; the three doors differ only in how much of it you operate. A consumer app hands you the top layer alone; an API hands you everything above the endpoint; open weights hand you the bottom layer and leave the rest for you to build. Illustrative — real stacks add layers, but the responsibility logic holds. (illustrative — source: Hugging Face Hub documentation — where open-weight models are published)

In production

Every major cloud offers the same model in more than one shape — and the shape, not the model, decides who runs the GPUs, what the meter counts, and who answers when it breaks.

AWS

On AWS, the API door is Amazon Bedrock: AWS operates the serving fleet, the meter counts input and output tokens, and capacity, patching, and availability are AWS’s problem. Take the open-weights door instead — the same family of models deployed on SageMaker endpoints or EC2 GPU instances — and the meter flips to compute time that accrues whether or not requests arrive, while scaling, inference software, and on-call become yours. Identical weights, opposite responsibility splits.

Azure

Azure AI Foundry puts both doors in one catalog. Deploy a model as a serverless API and you get token metering with the infrastructure hidden; deploy the same open-weight model onto managed compute and the meter becomes the GPU capacity you reserved, idle or busy, with the operational surface handed to you. The catalog entry looks the same — the deployment mode determines which bill and which pager you signed up for.

Google Cloud

On Google Cloud, Vertex AI’s managed model APIs meter tokens while Google runs the serving fleet; pull open weights from Model Garden onto your own Vertex endpoints or GKE GPU nodes and the meter becomes accelerator time. The economics invert at that boundary: token metering costs nothing at zero traffic, while self-hosted GPUs cost the same idle as busy — the fundamental trade every self-hosting decision starts from.

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.