Quotas and noisy neighbours
Lesson 4 of 5 in Multi-Tenant Isolation: One Platform, Many Customers, Zero Bleed.
Isolation is not only about data. A tenant who cannot read your other customers’ records can still take them down, and agents make the noisy-neighbour problem sharper than any workload before them: the amount of work per request is chosen by the model at runtime. A normal API request costs what its handler costs. An agent request costs whatever the loop decides — three tool calls or three hundred, one retrieval or a recursive crawl.
So the same non-determinism that gives agents their value gives you an unbounded cost and capacity variable per request, per tenant. One customer’s badly worded goal, one retry storm, one agent that discovers it can call itself, and your shared model quota, your tool upstreams, and your worker pool are all consumed by a single tenant while everybody else waits.
| Budget dimension | Unit and window | What the runaway looks like | Enforcement point |
|---|---|---|---|
Loop iterations | Turns per run | The agent oscillates between two tools, or re-verifies its work forever, and never reaches its stopping condition. | The harness loop itself: a hard turn cap, plus a no-progress detector that stops repeats rather than waiting for the cap. |
Tokens and spend | Tokens per run; currency per tenant per day | A context that grows every turn as tool output is appended, so cost per turn rises quadratically across a long run. | A metered budget object attached to the run, checked before each model call, plus a per-tenant daily ledger that shuts new runs out when breached. |
Tool calls | Calls per run, per tool | Twelve thousand search calls in one run; a write tool invoked repeatedly because the agent cannot tell it already succeeded. | The tool gateway: per-tool call ceilings per run, plus idempotency keys so repeats do not duplicate effects. |
Concurrency | Concurrent sessions per tenant | One tenant’s batch job opens 500 sessions and starves every interactive user in the pool. | Admission control before session creation: a per-tenant concurrency cap and a queue, never a first-come-first-served global pool. |
Wall clock | Seconds per run; session lifetime | A run parks on a slow upstream and holds compute, memory, and a licence seat for hours. | Timeouts at every layer — see agentops/reliability-plumbing — plus the platform’s own session lifetime as a backstop, not as your primary control. |
Shared upstream capacity | Requests or tokens per minute against a provider or internal API | One tenant consumes the whole model deployment’s throughput; everyone else gets rate-limit errors that look like your platform failing. | Per-tenant token buckets in front of the shared client, with reserved headroom for interactive traffic — and separate deployments or keys for your largest tenants. |
Containment ladder for a runaway tenant
- Tenant-scoped kill switch — rare, decisive, one tenant only
Pause new runs for this tenant — and only this tenant — while a human investigates. The point of a per-tenant kill switch is that the blast radius of your own emergency control is one customer, not the platform. If your only stop button halts everyone, a single bad tenant forces you to choose between an outage and an incident.
- Hard budget stop — the run ends, mid-task
The run’s metered budget — turns, tokens, spend, tool calls — is exhausted, so the harness terminates it and reports a truncated result. Make the stop explicit in the output and the trace, because a silent stop looks to the tenant like your platform quietly failing.
- Degrade and defer — keep serving, cheaper
Before killing work, remove expense: a smaller model for background tasks, expensive tools disabled, batch runs deferred to off-peak, retrieval breadth reduced. Degrading protects the neighbours while leaving the noisy tenant a working — if slower — service.
- Concurrency cap and fair queue — always on
A per-tenant concurrency limit plus weighted fair queueing so the scheduler serves tenants round-robin rather than in arrival order. This is the control that turns “one tenant filled the pool” into “one tenant filled their share of the pool”.
- Metering and alerting — the foundation everything else needs
Every run emits tenant-tagged counters for turns, tokens, tool calls, latency, and spend, and thresholds alert before limits bite. Without this layer the tiers above cannot be enforced or even explained to the customer — and you learn about the runaway from your invoice.
None of these mechanisms are new: timeouts, retries with backoff, idempotency keys, circuit breakers, and backpressure are the standard plumbing covered in agentops/reliability-plumbing. Multi-tenancy adds exactly one requirement to all of them — a tenant dimension. A global breaker protects your upstream and punishes the innocent; a per-tenant breaker isolates the fault to its source. A global rate limiter enforces capacity; a per-tenant limiter enforces fairness.
Two habits make the difference in practice. Attach a budget object to the run, not to a config file: turns, tokens, spend, and tool calls decremented in one place, so “out of budget” is a single well-tested code path rather than five scattered checks. And price the tenant’s tier into the budget, so quotas are a product decision your support team can explain, not an opaque throttle a customer discovers during a demo.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.