A few weeks ago, while reviewing our agentic application dashboard, I noticed a spike in LLM usage that didn’t align with any corresponding increase in traffic—it had remained flat for weeks. Initially, we suspected a logging bug, but it turned out to be a real issue.
The spike coincided almost exactly with an architectural change we’d shipped: migrating a portion of our pipeline from a single-agent to a multi-agent setup. Different agents now handle distinct parts of a task, coordinated through LangGraph, with a supervisor node deciding what happens next. On paper, this was a clean win—improving task decomposition, with each agent specialized in one thing rather than one model handling everything in a sprawling prompt. We expected usage to rise somewhat; more calls per task is the cost of that architecture, and nobody was pretending otherwise. But to our surprise, usage had roughly tripled for tasks that hadn’t functionally changed at all.
What’s in this article
- The intuition that stopped working
- Chasing the wrong suspect
- The fix that looked right and wasn’t enough
- What actually worked
- What the numbers looked like once we stopped guessing
- What I’d still reconsider
The intuition that stopped working
If you’ve only ever run a single LLM call per task, your mental model of cost is roughly linear: longer input, longer output, more tokens, more spend. You can eyeball it and be close enough.
That intuition falls apart the moment you introduce orchestration—and it falls apart quietly, which is worse than falling apart loudly. A supervisor agent now makes a decision about what happens next, and that decision itself costs tokens. Each sub-agent carries its own system prompt, its own tool schemas, and often its own copy of context the previous agent already had. None of this shows up as “more work being done”; from the user’s side, the task is still the same. It shows up as more machinery wrapped around the task, and machinery isn’t free just because it’s invisible to the user.
We hadn’t budgeted for it. Our architecture reviews focused on decomposition quality and correctness—did each agent do its job well, did the handoffs make sense—and not once, as far as I can remember, on what the token graph would actually look like once live and taking real traffic. That’s on us, and it’s probably on most teams making this move for the first time.
Chasing the wrong suspect
My first assumption was fan-out. Somewhere, the supervisor was spawning more sub-agent calls than the task genuinely needed, and trimming the branching logic would bring the number back down. It felt like the obvious place to look: more agents, more calls, simple arithmetic.
So I spent the better part of a day instrumenting call counts per supervisor decision, expecting to catch some node calling three agents when one would have done the job. I found almost nothing. The fan-out was doing roughly what it was meant to do, branch for branch.
That was frustrating in the specific way that being wrong about an obvious answer is frustrating. You’ve burned a day, and you’re back where you started, with one hypothesis eliminated and no replacement.
The replacement turned up almost by accident while reading through raw call logs for anything unusual rather than testing a theory. One agent’s output would fail a validation step—a malformed tool call, a schema mismatch, occasionally a model deciding to explain itself in prose instead of just calling the tool—and the retry wrapper would silently re-invoke that agent. Silently, because it completed successfully on the retry, so nothing downstream ever saw a failure. That sounds fine until you notice what re-invoking that agent actually required: rebuilding its upstream context, which in some paths meant re-running a sub-agent that had already produced a perfectly good result, purely to reconstruct the input the failing agent needed. One validation failure, three layers deep in the graph, could trigger a cascade of redundant calls, inflating token usage far beyond the apparent number of task executions.
The realization shifted my focus. The issue wasn’t over-branching in the supervisor logic; it was the hidden cost of retry mechanisms interacting with context dependencies. Each retry wasn’t just a single call—it was a chain of dependent calls, each with its own token overhead. In a single-agent setup, a retry is just one extra call. In a multi-agent graph, it can be a tree of calls, each rebuilding context from scratch.
The fix that looked right and wasn’t enough
Our initial fix was to add a simple caching layer. We cached the outputs of sub-agents at each node, so that if a validation failure occurred downstream, the retry wouldn’t need to re-run upstream agents; it could pull the cached result. This seemed like the obvious remedy—eliminate redundant computation.
It helped, but only marginally. The token bill dropped from 3× to roughly 2.4× the original, still far above the baseline. The reason became clear when we dug deeper: the retries were only part of the problem. The larger issue was that even without failures, the multi-agent structure itself was inherently token-hungry. Each agent carried its own system prompt, tool schemas, and context, and the supervisor’s decision-making added another layer of tokens on every task. Caching only addressed the retry cascade, not the baseline overhead of the architecture.
We needed a different approach.
What actually worked
We reduced the token overhead by rethinking how context flowed through the graph. Instead of each agent carrying a full copy of the upstream context, we introduced a shared context store—a lightweight, structured memory that agents could reference by pointer rather than by value. This cut down the duplication that had been inflating every call.
We also made the supervisor more efficient. Rather than having it make a full decision on every task, we implemented a fast-path rule-based triage for routine cases. Only when a task required nuance did the supervisor step in with a full LLM call. This preserved the flexibility of the multi-agent design while eliminating the overhead for the majority of tasks.
Finally, we addressed the retry cascade with a targeted fix: instead of a blanket retry wrapper, we added validation at each node to catch malformed outputs before they propagated. This reduced failures by over 70%, and when failures did occur, the shared context store meant retries didn’t need to re-run upstream agents—they could reconstruct the input from the store.
What the numbers looked like once we stopped guessing
After implementing these changes, we tracked the usage for two weeks. The token bill dropped to roughly 1.4× the original single-agent baseline—a significant improvement from 3×. More importantly, the cost per task became predictable. We now estimate token usage based on task complexity, and we’ve built a dashboard that breaks down spending by node, so we can spot anomalies before they become budget surprises.
The key takeaway: multi-agent architectures are powerful, but they come with hidden token costs that don’t show up in simple call counts. Our initial 3× spike was a wake-up call, and the process of diagnosing it taught us more about orchestration than any documentation could.
What I’d still reconsider
Even now, I’m not fully satisfied with the 1.4× overhead. In 2026, as LLM costs continue to drop and context windows grow, the balance might shift further toward multi-agent designs. But for now, I’d reconsider whether every task genuinely needs orchestration. Some tasks are simple enough for a single well-prompted model, and forcing them through a graph just adds cost.
I’d also revisit the retry strategy. Our current approach is better, but it still allows for some redundant calls in edge cases. A more sophisticated approach might involve using the model’s own confidence scores to decide whether a retry is worth the cost.
Finally, I’d push for more discipline in architecture reviews. We now include token estimation as a first-class consideration when designing any new feature with agentic components—right up there with correctness and latency. It’s a small change in process that can prevent a 3× bill from showing up unannounced.