Prompt, hosted, ephemeral: three ways to run an agent

Lesson 2 of 5 in Microsoft Foundry Agent Service, Three Ways to Run an Agent.

Here is the decision that shapes everything else on this platform. Foundry Agent Service documents exactly two agent types — prompt agents and hosted agents — and then documents a third pattern that creates no agent at all: calling the Responses API directly, which the docs call an ephemeral agent.

The distinction is not a tier list. It is a question of where the agent’s definition lives: in Foundry configuration, in your container, or in your application code. Everything downstream — cost, scaling, identity, publishing, what you can version — follows from that one answer.

Prompt agents

Declarative. No infrastructure.

A prompt agent is defined entirely through configuration — instructions, model selection, tools — with no application code to maintain. Microsoft’s wording is blunt: no containers or packages to optimize, scale, or monitor for security.

Two authoring routes, same resource:

Portal-first — build it in the Foundry portal, iterate on instructions and tools in a UI.
Code-first — create and update the same agent via the SDKs or REST API, which is how it belongs in CI/CD alongside everything else you deploy.

Foundry owns the loop: the model call, the tool orchestration, conversation persistence, scaling with request volume. You own the instructions, the tool selection, and the evaluation.

The trade is control. If your agent needs a custom planner, a framework-specific graph, a bespoke retry policy, or code between tool calls, there is nowhere to put it.

Hosted agents

Your code. Their endpoint, scaling, and identity.

You ship a container image — or a .zip of source that Foundry builds into an image — and Foundry runs it with a managed endpoint, automatic scaling, a dedicated Microsoft Entra identity per agent, session-level state persistence, and end-to-end observability.

Frameworks Microsoft documents as supported: Microsoft Agent Framework, LangGraph, the OpenAI Agents SDK, the Anthropic Agent SDK, the GitHub Copilot SDK, or custom code of your own.

Your container exposes one or more protocols — Responses, Invocations, or Invocations (WebSocket) — each provided by a lightweight library that handles the HTTP or WebSocket server, health checks, and OpenTelemetry integration. The Invocations protocol exists for webhooks and non-OpenAI-shaped payloads.

Inside the container, your code reaches catalog models and platform tools (file search, code interpreter, web search, plus SharePoint, WorkIQ and Fabric IQ) through the Responses API on the project endpoint — one entry point for inference and tool orchestration.

Ephemeral agents

No agent resource at all.

Call the Responses API directly from your own application and the agent’s definition — instructions, tools, model — lives in your code rather than as a persisted Foundry resource. There is nothing to create, update, or delete in Foundry.

What you still get through the project endpoint: catalog models, platform tools, project-scoped data, On-Behalf-Of auth, and project-level observability and governance. This is not an unmanaged back door; it is the platform without the agent object.

What you give up follows from the same fact. No persisted agent means nothing to snapshot as a version, nothing to publish to Teams, nothing to expose as an A2A endpoint, and no per-agent identity to grant permissions to — every platform feature that hangs off an agent resource has no hook.

Best fit: an existing application that needs model calls with governed tools, not a long-lived autonomous worker.

The three ways, side by side (rows drawn from Microsoft’s compare-agent-types table where marked)
Agent type	Authoring surface	Compute	Cost model	Best for
Prompt agent	Configuration only — instructions, model, tools. Portal-first or code-first via SDK/REST.	None to manage; scales with request volume under Foundry management.	Per-call inference + tool usage (per Microsoft’s compare table).	Assistants and task agents whose logic is instructions plus tools; teams without a container pipeline.
Hosted agent	Your code, as a container image or a .zip Foundry builds. Agent Framework, LangGraph, OpenAI Agents SDK, Anthropic Agent SDK, GitHub Copilot SDK, or custom.	Foundry-managed container compute; scales container instances per session and request volume. Sessions run in per-session VM-isolated sandboxes.	Per-call inference + tool usage + container compute (per Microsoft’s compare table).	Custom control flow, framework-specific graphs, long stateful tasks, code between tool calls.
Ephemeral (Responses API)	Your application code. No agent resource exists in Foundry.	Whatever already runs your application — Foundry runs the model call and platform tools, not your process.	Not in the compare table: you pay for what you call through the project endpoint, and your own hosting stays your own bill. Check the pricing page.	Adding governed model calls and tools to an existing app; short request-scoped work.

Microsoft Learn — What are hosted agents? Sessions and state store (checked 2026-09)

Hosted agents run in per-session VM-isolated sandboxes. Each session gets a dedicated sandbox with a persistent filesystem ($HOME and /files), which is what allows scale-to-zero with stateful resume: state is restored automatically when an idle session wakes up.

The numbers to hold: session idle timeout is configurable from 2 to 60 minutes, defaulting to 15; on idle the platform deprovisions compute and persists the filesystem; a session is permanently deleted after 30 days of inactivity.

Alongside the filesystem there is a durable server-backed key-value state store of keyed JSON items that persists independently of compute — it survives crashes, restarts and idle eviction, has a default 30-day item idle window that writes renew (and can be configured never to expire), and can hold framework checkpoints for LangGraph or Agent Framework. It can also partition items per end user for multitenant agents.

Two vocabulary traps in that spec, because they cost people days.

Sessions are not conversations. A session is a hosted agent’s compute plus its filesystem — ephemeral by design, reclaimed on idle, gone after 30 days of silence. A conversation is the durable item history stored in Foundry. Put anything you need next quarter in the state store or a conversation, never in /files and never in process memory.

Idle eviction is not failure. Your container will be stopped mid-life and resumed later with its filesystem intact and its process state gone. That is the platform working as designed, and it is why frameworks with explicit checkpoints — LangGraph, Agent Framework — sit comfortably here while a hand-rolled loop holding everything in local variables does not.

Which of the three ways should this workload use?

Interactive decision tree — outcomes:

Prompt agent
Declarative agent, no container to patch or scale, and it is a first-class resource: versioned, publishable, identity-bearing. Author it code-first so it lives in source control rather than in a portal nobody can diff.
Hosted agent
Ship your container (or a .zip) and take the managed endpoint, per-session sandbox, autoscaling and per-agent Entra identity. Budget for the extra cost line — container compute on top of inference and tools — and design for idle eviction from day one.
Ephemeral — call the Responses API
No agent resource, nothing to manage, and you keep catalog models, platform tools, On-Behalf-Of auth and project-level observability. Accept the consequence: nothing to version, publish, or hand another agent. If someone later asks “which version of the agent did that?”, you will be answering from your own deployment history.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.