Multi-Agent AI Systems: When One Agent Is Not Enough
A second agent isn't a feature, it's infrastructure you now maintain and pay for. When that trade genuinely pays off, and the honest cases where it doesn't.
Anthropic's own agent-infrastructure team has admitted to spending months building an elaborate multi-agent architecture for a task, only to later discover that a single, well-prompted agent achieved the same result. That's not an embarrassing story to bury. It's the single most useful data point in this entire topic: the team that builds multi-agent systems for a living found out the hard way that a second agent is not automatically an upgrade.
The one sentence to remember
A second agent isn't a feature you add for free. It's infrastructure you now have to build, coordinate, monitor, and pay for, and it only earns that cost when a single agent's actual constraints, not its raw capability, are the real bottleneck.
This is an architectural look at multi-agent systems done properly: the supervisor-worker pattern, how agents actually communicate and share state, delegation and parallel execution, what happens when a worker fails, and, because this is the question that actually separates good engineering judgment from following a trend, exactly when you shouldn't reach for this pattern at all.
The Architecture
The supervisor doesn't do the domain work itself. It delegates to a set of specialized workers, each with a narrow job:
| Worker | Scope |
|---|---|
| Research Agent | Gathers and synthesizes information from external sources |
| Coding Agent | Writes and modifies code against a defined task |
| Testing Agent | Verifies the coding agent's output actually works |
| Security Agent | Reviews changes for vulnerabilities before anything ships |
Why this shape, specifically
Each worker holds a narrower, cleaner context than a single agent trying to research, code, test, and audit security all at once ever could. That's the real value multi-agent architecture is selling, not more intelligence, a cleaner working context per task.
Supervisor Agents
The supervisor's job is orchestration, not execution: interpret the user's actual goal, decompose it into tasks that map to a specific worker's specialty, and synthesize the workers' individual results into one coherent final answer. A supervisor that also tries to do the coding or the research itself has stopped being a supervisor and started being a bottleneck with extra steps.
A well-scoped supervisor holds remarkably little domain knowledge on purpose. It knows who handles what, and it knows how to combine results, not how to do the underlying work itself.
Worker Agents
A worker agent should be narrow enough that its context stays focused on exactly one kind of task. This is the same context-management principle from AI Agents Are Not Chatbots, applied at the system level instead of inside a single agent's loop: a worker that only ever sees research tasks builds a cleaner, more focused context than a generalist agent juggling four different jobs in the same conversation.
Scope the permissions as narrowly as the task
A Security Agent reviewing code for vulnerabilities has no legitimate reason to hold write access to the repository it's reviewing. A Research Agent gathering external information has no reason to hold credentials for your production database. This is the same least-privilege principle from AI Agent Identity, and multi-agent systems are exactly where it matters most, because a compromised or malfunctioning worker's blast radius should be limited to what that specific worker actually needed.
Agent Communication
Workers and supervisors need to exchange more than free-form chat messages to stay coordinated at scale. A structured message, task description, expected output format, relevant context handed down from the supervisor, is what keeps a worker from having to guess at scope, and what keeps the supervisor able to parse a worker's result programmatically instead of re-reading prose to figure out what happened.
When one worker's output needs to flow into another agent entirely, the identity chain matters too: the delegation should carry forward who originally asked and which agents have acted since, the same actor-chain principle covered under Agent-to-Agent Identity in AI Agent Identity, so a result three hops deep in the system is still traceable back to its origin.
Shared Memory
Multi-agent systems face a version of the memory question from AI Agents Are Not Chatbots, with an added wrinkle: memory now has to be shared safely across multiple, independently acting agents, not just persisted across turns of one.
| Pattern | How it works | Risk |
|---|---|---|
| Private per-agent context | Each worker holds only what it needs for its own task | Coordination requires the supervisor to explicitly pass information between workers |
| Shared blackboard memory | All agents read and write to a common store | Two agents writing conflicting updates to the same shared state, or one agent's irrelevant output polluting another's context |
Shared memory is where context pollution actually happens
A shared store that every worker writes into freely tends to accumulate exactly the problem multi-agent architecture was supposed to solve: one agent's noisy, task-specific output leaking into another agent's context and degrading its focus. Scope what gets written to shared memory as deliberately as you'd scope a database write in any other multi-service system.
Delegation
The supervisor's delegation decision can be as simple as rule-based routing, keyword or task-type matching to a fixed worker, or as flexible as letting the supervisor model itself reason about which worker fits a given sub-task. The flexible approach handles novel requests better; the rule-based approach is cheaper, faster, and far easier to debug when something routes wrong.
Match the delegation mechanism to how predictable the task shape actually is
If your incoming tasks fall into a small, well-known set of categories, rule-based routing to the right worker is simpler to build, cheaper to run, and easier to reason about than paying for a model call just to decide which of four workers should handle something a keyword match could have resolved instantly.
Parallel Execution
This is where multi-agent architecture earns real, measurable value, when independent sub-tasks can genuinely run at the same time instead of waiting on each other. A Research Agent gathering background information and a Security Agent scanning existing code for known issues can run concurrently if neither one's output depends on the other's.
Fake parallelism costs money without buying anything back
Running four agents "in parallel" whose tasks are actually sequentially dependent, the Testing Agent can't meaningfully test code the Coding Agent hasn't written yet, doesn't buy you speed. It buys you four times the token spend for work that still has to happen in order. Multi-agent parallelism is a token-spending strategy: you're paying to buy real concurrency, and that trade only pays off when the task genuinely decomposes into independent threads.
Failure Handling
A single agent that fails mid-task fails the whole request. A multi-agent system has more failure surface, more agents that can individually break, but also more room to degrade gracefully if it's actually built for it.
Decide per-worker: retry, skip, or escalate
A Research Agent that times out might be worth retrying once. A Security Agent that fails should probably block the whole pipeline rather than being silently skipped, given what skipping it actually means.
The supervisor needs to know the difference between a worker's failure and a worker's disagreement
A worker returning "I couldn't complete this" is a failure to handle. A worker returning a result that conflicts with another worker's output is a coordination problem, not a failure, and needs a different response: reconciliation, not a retry.
Apply the same retryable-versus-not distinction from single-agent systems
A transient failure in one worker is worth a retry. A worker consistently failing on a specific input needs escalation to a human or a different approach entirely, the same principle covered under Failure Recovery in AI Agents Are Not Chatbots, now applied per-worker instead of per-tool-call.
Agent Coordination
The hardest problem in multi-agent systems isn't any individual agent, it's reconciling what happens when their outputs don't cleanly agree. A Coding Agent ships a fix. A Security Agent, working from a slightly different context, flags a vulnerability in code the Coding Agent already believes it resolved. Someone, or something, has to decide whose view wins, and that arbitration logic belongs explicitly in the supervisor, not left as an implicit assumption that workers will somehow never disagree.
Design for disagreement, don't just hope it won't happen
Multiple specialized agents looking at overlapping parts of the same problem from different angles is exactly why the architecture is valuable, and exactly why conflicting conclusions are a normal, expected output, not an edge case. Build the supervisor's synthesis step to actually handle disagreement, not just concatenate whatever the workers returned.
When Should You NOT Use Multi-Agent Architecture?
This is the question that actually matters most, and Anthropic's own agent-infrastructure team has published a direct answer worth taking seriously: most production AI systems don't need another autonomous agent. They need a workflow with clear steps, tight tools, and measurable outcomes.
The three conditions where multi-agent genuinely earns its cost
Anthropic identifies exactly three situations where multiple agents reliably outperform one: when a single agent's context gets polluted trying to hold too many kinds of work at once, when the underlying tasks can actually run in parallel, and when specialization measurably improves tool selection or task focus. Outside those three, coordination costs typically exceed whatever benefit you were hoping to get.
| If your situation is... | The right architecture is probably... |
|---|---|
| A single, well-defined task with a predictable sequence of steps | A deterministic workflow, no agent required at all |
| One agent, one focused context, that can hold the whole task without getting overloaded | A single agent, possibly with better prompting before you add anything else |
| Context genuinely gets overloaded juggling distinct kinds of work | Multiple agents, split along that real boundary |
| Independent sub-tasks that can actually run concurrently | Multiple agents, parallelized for real, measurable time savings |
| Tasks that benefit from genuinely different tool access or specialized focus | Multiple agents, scoped narrowly to their specialty |
Remember the actual anecdote this article opened with
A team that builds agent infrastructure for a living spent months on a multi-agent architecture before discovering a single agent with a better prompt matched it. If that team can make that mistake, "we're building multiple agents" is worth defending with a specific answer to "which of the three conditions does this actually satisfy," not assumed as obviously the more sophisticated choice.
The Bottom Line
Multi-agent architecture is a real, valuable pattern for a real, specific set of problems: context that would otherwise get overloaded, work that genuinely parallelizes, and tasks that benefit from real specialization. It is not a more advanced version of a single agent that you should reach for by default because it sounds more sophisticated. Every additional agent is additional coordination logic, additional failure surface, additional cost, and additional places for two reasonable-looking outputs to quietly disagree with each other.
The question to ask before adding a second agent
Which of the three conditions, context pollution, genuine parallelism, or real specialization, does this specific task actually satisfy? If you can't name one clearly, the honest next step isn't a supervisor and four workers. It's a better prompt on the one agent you already have.
One agent, done well, beats four agents coordinating badly. The team that builds this stuff professionally learned that the expensive way, so the rest of us don't have to.
For a concrete instance of this architecture, a Planner, a Worker, and an independent Reviewer, built step by step with real code, retries, and an escalation path, see Build a Multi-Agent System.
Written by
Chetan Yamger
Cloud Engineer · AI Automation Architect · Modern Workplace Consultant
Cloud Engineer, AI Automation Architect, and Modern Workplace Consultant based in Amsterdam, Netherlands. Specializing in scalable, secure enterprise solutions with Microsoft Azure, Intune, PowerShell, and AI-driven automation using ChatGPT, Gemini, and modern LLM technologies.
Stay in the loop.
New articles, straight to you.
Deep-dive technical articles on Intune, PowerShell, and AI — no noise, no spam.
Discussion
Share your thoughts — your email stays private
Leave a comment