Cost has a shape — and how to run an honest bake-off

AWS — session lifecycle

AgentCore Runtime offers two substrates. microVM sessions last up to 8 hours, terminate after 15 minutes of inactivity (documented defaults — check current Service Quotas), and treat session state as ephemeral: durable context is supposed to live in AgentCore Memory. Instance sessions run on isolated instances from a capacity provider for up to 14 days, with persistent volumes that survive stops and re-attach on resume with the same session id.

Cost consequence. CPU is not charged while nothing runs, but memory is billed for the life of the session — so a long-lived session parked waiting on a human is not free. Design consequence. If your agent must survive days of back-and-forth, that is an explicit substrate choice, not a default.

Microsoft Foundry — session lifecycle

Hosted agents run in per-session VM-isolated sandboxes with a persistent filesystem, and the platform is built around scale-to-zero with stateful resume: the idle timeout is configurable from 2 to 60 minutes (default 15), after which compute is deprovisioned and session state persisted, then restored when the session resumes. Sessions are permanently deleted after 30 days of inactivity. A separate durable key-value state store holds keyed JSON items independently of compute — it survives crashes, restarts and idle eviction, with a default 30-day item idle window that writes renew, and it can hold framework checkpoints for LangGraph or Agent Framework.

Cost consequence. Bursty, human-paced workloads pay less because compute genuinely goes away. Design consequence. Your resume path has to work, and it is the platform’s state store — not your own database — that makes it work unless you decide otherwise.

Google Cloud — session lifecycle

Agent Runtime bills allocated vCPU and RAM rounded to the nearest second, and the pricing page states that for Runtime, idle time waiting for the next prompt between turns is not billed. Sessions hold short-term conversational state; Memory Bank derives long-term memories from those sessions using generative models. There is also a monthly per-account free tier — the first 50 vCPU-hours of Agent Compute, 100 GiB-hours of Agent Memory and 1 GiB-month of Agent Storage.

Cost consequence. Turn-based conversational agents look cheap here, and the free tier will make your prototype look even cheaper than it is. Design consequence. Exclude the free tier from any bake-off arithmetic; a pilot that fits inside it tells you nothing about the shape of the production bill.

1 · Fix the task set before you touch a console

Take 100–300 real tasks with known-good outcomes — your golden dataset — and freeze it. If the task set changes between platforms, you have measured nothing. This is also the moment to notice that the eval suite is the reusable asset here: it outlives the bake-off, the platform choice, and probably the framework.

2 · Hold the model class constant, then vary it deliberately

Run the same model family on every platform if you can — several are available in more than one place, and Foundry documents swapping catalogue models without changing agent code. Then run a second pass with each platform’s house model, because that is a real option you would exercise. Reporting one number that silently mixes model choice with platform choice is the single most common way bake-offs mislead.

3 · Measure cost per resolved task, not cost per run

A platform that fails 20% of tasks and retries is not cheaper. Divide total spend — inference, platform SKUs, retrieval, observability ingestion — by the number of tasks that actually met the success criterion. Then report p50 and p95 latency separately, because the tail is what users experience and what your timeout policy has to survive.

4 · Reproduce your real traffic shape, especially idleness

Bursty human-paced traffic and steady batch traffic land on completely different parts of these billing models, given how differently the three treat waiting and idling (see the tabs above). If your agents sit waiting for approvals, simulate the waiting. If they run overnight in batches, run overnight batches.

5 · Exclude free tiers and promotional credits

Google Cloud publishes a monthly per-account free tier for Agent Compute, Memory and Storage; vendors also hand out proof-of-concept credits. Both are real money and neither is a rate. Model the bill at your projected production volume, then note the free tier as a one-line footnote — never as part of the comparison.

6 · Count the platform-team hours, and count them at their real price

Time to first working deployment, time to wire identity, time to get a trace end-to-end, time to satisfy your security reviewer. These hours are usually a larger delta than the compute rates, and they compound: a platform inside your existing landing zone starts with account vending, networking and policy baselines already solved.

7 · Write down what would flip the result

Finish with two or three named conditions that would change the recommendation — a 10× volume increase, the delivery surface moving off Teams, the primary corpus moving to another cloud, an egress control leaving preview. This turns a snapshot into a decision you can revisit deliberately, which is exactly what the deployment and versioning module wants from you.