A multi-agent design is useful when different branches need genuinely different instructions, tools, data access, or ownership. It is not a default upgrade from a single agent. Every handoff adds latency, cost, state transfer, and another place where intent can be lost. Begin with a deterministic workflow and one bounded agent, then split only the stages whose evaluations show a clear benefit from specialization.
Write the workflow contract before naming agents
Define the input event, terminal outcome, authorized side effects, evidence required at each transition, retryable failures, approval points, and escalation owner. Mark which steps are deterministic code, which require model judgement, and which require human authority. This prevents an architecture diagram from disguising an undefined business process.
Choose manager ownership or a true handoff
Use a manager pattern when one agent should retain responsibility for the final answer and call specialists as bounded tools. Use a handoff when the specialist should become the active owner of the next interaction. Keep each route description narrow and test ambiguous cases. Do not pass every specialist the full tool inventory or complete conversation history when a filtered task packet is sufficient.
Split an agent only when the new boundary improves instructions, authority, context, or evaluation—not because more agents look more capable.
Keep durable state outside model conversation
Store workflow ID, current state, input snapshot, specialist assignment, tool results, approvals, attempts, and terminal outcome in an execution ledger. The model can propose the next transition, but application code should validate and commit it. Resume from the last completed checkpoint after a timeout rather than replaying the entire conversation and repeating side effects.
Use typed handoff and tool contracts
Give every specialist a versioned input schema with required evidence and an output schema that distinguishes decision, confidence, citations, requested action, and unresolved fields. Validate types and business constraints at runtime. Return structured, repairable errors for missing fields, but cap correction loops and route persistent failures to an operator instead of letting agents negotiate indefinitely.
Enforce authority outside the model
An auditor agent can catch inconsistencies, but it is not an independent security boundary when it relies on the same data and similar model behavior. Enforce tenant access, allowed transitions, monetary limits, consent, record state, and approval tokens in deterministic middleware. Use a human approval for consequential exceptions. Give the approving party the proposed action, evidence, diff, and rollback consequence—not merely another agent's confidence score.
Make retries safe across agent boundaries
Assign an idempotency key to each business action and record tool-call outcome before advancing state. Retry only transient failures, with a maximum attempt or elapsed-time budget. A specialist timeout should not cause the manager to call a second specialist that performs the same write unknowingly. Compensating actions must be explicit workflow states with their own authority checks.
Pick the orchestration pattern the workload actually needs
Four patterns cover most business operations, and the choice is about control flow rather than agent count. The manager-and-tools pattern keeps one coordinating agent that calls specialists as functions; it is simplest to test and debug because state never leaves a single owner, and it fits retrieval, classification, and drafting where the coordinator stays accountable for the answer. The handoff chain passes conversation ownership from a triage agent to a specialist; it fits deep, single-task interactions such as booking flows where the specialist needs the full interaction context. The parallel fan-out splits independent sub-tasks across agents and merges results; it fits analysis work like reviewing multiple documents or comparing quotes, provided each result carries its own evidence and the merge logic is deterministic code. The event-driven network routes messages between agents on a shared bus; it fits always-on operations where agents respond to external events independently, but it requires the strongest execution ledger because no single agent holds the whole state. A single business may legitimately use different patterns per workflow; forcing one pattern everywhere is as fragile as scattering handoffs everywhere.
Guard against the loops that silently consume the budget
The most expensive multi-agent failures are not wrong answers but unbounded repetition. Three loops recur in production and each needs an explicit guard. The correction loop happens when specialist validation keeps rejecting a payload and the producing agent keeps guessing; cap retries with a hard maximum, then route to an operator with the failed payload and rejection reasons. The escalation loop happens when no specialist claims ownership and the routing agent keeps querying others with no decision rule; require a default unowned queue and alert on records sitting in it. The retry loop happens when a specialist timeout causes duplicate writes; it is already covered by idempotency keys but must be tested under real latency, because a loop that only appears at production concurrency is the kind that arrives with a cloud bill. Measure correction-loop rate, average handoffs per task, and idle agents per hour as standing metrics; when handoffs per task climbs while task success stays flat, the topology is adding motion rather than value.
A worked pipeline: the intake flow that justifies two agents
Consider a lead pipeline that must qualify, enrich, route, and follow up. A single agent asked to do all four produces acceptable-looking text but leaks evidence between stages, because the qualification judgement bleeds into the routing choice without a record of why. The split version has a classifier agent that outputs a structured qualification record with field-level confidence and citations from the enquiry itself, and a routing agent that consumes only that record plus policy rules to choose an owner and produce an actionable task packet. The execution ledger stores each record and decision, application code validates the schemas, and a two-week evaluation compares the split against the single-agent baseline on route accuracy, follow-up start time, and operator corrections. If the split version does not beat the baseline, the added boundary is not yet justified and the single agent with structured output is the better production system; the same evaluation applies in reverse when a single agent is failing at a boundary that a specialist would isolate.
Evaluate routing and end-to-end outcomes
Test wrong-route, no-route, conflicting specialist results, stale context, malformed handoff payload, tool denial, partial completion, repeated event, approval rejection, and operator takeover. Track route accuracy, task success, unauthorized-action attempts, handoff count, correction loops, tool errors, latency, token cost, human intervention, and rollback rate. Compare the multi-agent version against the simpler baseline; keep the additional topology only where it measurably improves the target outcome.
