Centralize the judgment. Distribute the work.

McKinsey published its State of AI survey on 25 August, 1,719 professionals and business leaders across industries. Two numbers from it belong next to each other. Among organizations above $1 billion in revenue, 40% say they are scaling AI agents, up from 27% a year ago. And 6% of respondents qualify as high performers, meaning they attribute at least 5% of EBIT to AI with significant impact. That second figure did not move. Neither did the 37% who report some EBIT impact at all.

So adoption ran and value did not follow it. The usual explanations are data quality, change management, and the fact that pilots are easier than production. All true, all discussed to death. There is a fourth one that nobody puts on a slide because it sounds like an implementation detail, and it is the one I keep finding when I open other people's systems.

The agent is one model. It reads the input, decides what to do, calls the tool, cleans the result, formats the answer, and checks its own work, and every one of those steps goes to the same frontier endpoint at the same price. One brain doing all the thinking, including the parts that require none. The same survey reports 20% saying AI operating costs have already constrained their use. That is the bill for the architecture, arriving before the value does.

Most of an agent run needs no judgment at all

Open the trace of any agent that has been in production for a quarter and count what the model is actually being asked to do. Read a form. Pull a field out of a paragraph. Decide which of four tools applies. Turn a messy API response into three keys. Check that a date is a date. Retry because the JSON came back with a trailing comma.

Then find the step where something genuinely ambiguous happens, where the input is underspecified and the cost of being wrong is real. There is usually one. Occasionally two.

Peter Belcak and colleagues at NVIDIA Research made this argument formally in Small Language Models are the Future of Agentic AI, and the observation underneath it is the useful part. Agentic systems consist of models "performing a small number of specialized tasks repetitively and with little variation". That is not a claim about capability. It is a claim about the shape of the work. An agent is not a conversation. It is a loop that runs the same six steps a thousand times a day with different arguments, and a general model is being paid general prices to do a narrow job it does not need generality to finish.

The reason this survives in production is that it never fails loudly. A single model that handles every step is the fastest thing to build, it works on day one, and its cost problem shows up as a line in a monthly invoice rather than as an incident. Nobody gets paged for spending too much per turn. The system just quietly stays too expensive to point at the volume that would have made it matter, which is exactly what "operating costs constrained our use" means when you translate it out of survey language.

Small got good enough while the architecture stayed the same

The counterargument used to be real. Two years ago the small open models could not hold a tool schema straight, and splitting the work meant babysitting a second class of failure for a saving that did not cover the engineering.

That changed and most teams did not rewrite anything. Google released Gemma 4 on 2 April under Apache 2.0, a family that runs from a 2B variant up to a 31B dense model, with 256K context and tool calling across the lineup, targeted explicitly at laptops, workstations and edge hardware. Qwen and the rest of the open field moved in the same direction over the same months. The claim I would defend is narrow and it is enough: for extraction, classification, normalization and formatting against a fixed schema, a well-chosen small model now clears the bar, runs in your own perimeter, and costs a fraction per call.

Which means the constraint moved. It is no longer whether the small model can do the step. It is whether anyone has bothered to name the steps.

Naming them is the work. You cannot route what you have not decomposed, and most agents are not decomposed at all. They are a long prompt with tools attached, and inside that prompt six different jobs are tangled together with no boundary between them. This is the same failure I wrote about when capabilities came before vendors: the missing primitive gets monkey-patched inside the agent, and a year later nobody can say where one responsibility ends and the next begins. A monolith you cannot describe is a monolith you cannot split, whatever the price of the alternative.

Distributing the work does not distribute the accountability

Here is where the enthusiasm needs a brake, because "many small models" is being sold as a cost story and it is really a governance story with a cost benefit attached.

One brain has one failure mode and one eval suite. Six models have six of each. If you split the loop and do not build the contracts, you have not decentralized intelligence. You have decentralized the debugging, and you will find out in production which of the six quietly started returning null for a field that three downstream steps assumed was always present.

So the rule I hold teams to is that a step only leaves the planner when it has three things. A named contract, meaning a fixed input shape, a fixed output shape and a stated quality bar. Its own eval set, meaning real cases with expected outputs, run on every model change. And an owner, a person, not a squad.

That third one is the one people skip and it is the one that decides whether this holds up. A worker model without an owner is a component that degrades silently for two quarters because the model behind it was updated and nobody was watching that particular seam. The centralized brain at least had a single throat to choke.

The upside of getting it right goes past the invoice. When each step is a named call with its own inputs and outputs, the trace stops being a wall of tokens and becomes a record you can read. Which step decided, on what input, against which version. I argued in August that an agent acting under a human's credential poisons the audit log at the source. The composition problem is the sibling of that one. A monolithic agent cannot tell you which part of itself made the call that a candidate will challenge next year, because there are no parts. A decomposed one answers with a line number.

What this looks like at Shakers on a Wednesday

Our Matchmaking flow starts with a brief written by a human in a hurry. Free text, often contradictory, usually missing the constraint that turns out to matter. Somebody writes "senior React" and means someone who has shipped a design system, or means someone cheap and available on Monday, and those are different searches.

Working out which one it is, from a paragraph that does not say, is judgment. It is underspecified, the cost of being wrong is a bad shortlist in front of a client, and it stays with the frontier model. I have no interest in saving money there.

Everything around it is not judgment. Normalizing a stack tag to our taxonomy. Pulling an availability window out of a sentence. Parsing a rate range written six different ways. Checking a profile against a hard filter. Shaping a structured result for the next step. Those calls run constantly, they have narrow inputs and narrow outputs, and the correct answer is checkable, which is the property that makes an eval set cheap to write and a small model safe to trust.

The same split runs through Alma and Talent Insight. One interpretive step where the model is deciding what a person's history actually says. A long tail of retrieval, extraction and formatting where it is not deciding anything, just doing the same transformation on new arguments.

The test I use when a team proposes moving a step off the planner is one question. Write down the expected output for twenty real inputs. If you can, the step has a contract and it can leave. If you cannot agree on the expected output, the step is judgment and it stays where the judgment is, and the argument about cost is premature.

Closing

Centralized and decentralized is the wrong frame for this, which is why the debate keeps going in circles. Nobody is choosing between one intelligence and many. The real choice is what gets centralized, and the answer is judgment, ambiguity, and the calls that are expensive to get wrong. Everything else is transformation work with a checkable answer, and transformation work belongs at the edge, close to the data, cheap, owned, and under its own eval.

The 6% in the McKinsey survey did not find a better model than everyone else. There is one shelf and we all shop on it. What separates them is that somewhere in their stack, somebody sat down and named the steps, and once the steps have names the rest is engineering.

The rest of the field is running a single brain that thinks just as hard about parsing a date as it does about the decision the business actually pays for. That works. It will keep working. It will also stay exactly as expensive at ten times the volume, which is the volume where the earnings number changes.