You have a few hundred typed fields to extract from a stack of real documents: amounts, dates, coverage limits—one structured value per field. The default approach is to send every field to a GPT-4-class hosted API. It works, but the bill becomes the largest line item in the pipeline’s running cost. Most of those fields are straightforward lookups that a far smaller model can handle easily, yet you’re paying flagship prices for all of them.
The obvious reaction is to swap in a small model and pocket the savings. Done blindly, however, it backfires. A model too weak for the task returns broken typed output or fumbles a transformation, and if nobody checks, the wrong value ships. So the answer is not “use the big model” and it is not “use the small model.” Instead, it is to choose the model the way earlier bricks choose a method: by criteria, then verify, and escalate only when the verification fails.
This article is a companion to Enterprise Document Intelligence, the series whose philosophy is laid out in Amplify the Expert. That series builds enterprise RAG from four bricks: document parsing, question parsing, retrieval, and generation. This piece develops one decision inside the generation brick: which model runs the call, and what happens when the cheap one is not good enough? It leans on Article 6C (dispatch), which reads a suggested model tier from the parsed question; Article 8C (validation), the signal that triggers escalation; and Article 10 (adaptive parsing), which shares the same start-cheap-escalate-on-demand shape applied to the parser.
Everything below is backed by a real benchmark: a field-extraction task covering a few hundred typed fields across dozens of documents, plus a focused sweep of twenty local models against one hosted flagship—all at temperature 0 with JSON output. The numbers are aggregate. No document, field label, or figure from that benchmark appears here.
!Where this companion sits: inside brick 8, generation, the fourth brick of Part II – Image by author
📓 The runnable notebook for this article is on GitHub: doc-intel/notebooks-vol1. It runs the same field through the cascade, prints the per-field cost and validation verdict at each rung, and shows escalation firing only on the fields the cheap local model gets wrong.
1. Two angles: cost, and the loop
There are two reasons to care about model selection, and they reinforce each other.
The cost angle is straightforward. Flagship models cost an order of magnitude more per call than a small local one. On an extraction job with thousands of field-level calls—or a corpus assistant answering thousands of questions a day—the model bill dominates the running cost. Most of those calls don’t need a flagship. Routing the simple ones to a cheaper model is money left on the table until you actually do it.
The loop angle is about correctness under that constraint. You cannot cut costs by guessing that a small model will cope. Instead, you start with a model, check whether its answer holds, and escalate to a stronger one only when the check fails. The loop is what makes the cost saving safe: the cheap model is the default attempt, not a gamble, because a failed attempt is caught and retried on a bigger model rather than shipped.
Cost pushes you down the model ladder. The loop stops you from falling off it. This cheap-first, escalate-on-failure shape has a name in the literature: an LLM cascade.
2. What the numbers say
The design in Section 3 rests on two measured facts. Take them first, because they are the reason the cascade is shaped the way it is.