When the first serious token invoice arrives, the response is predictable: identify the heavy users, cap them, and ask the vendor for a discount. All three actions are available. None of them is where the bill is decided.
Token spend is determined by architecture — by decisions made in the platform layer before any user types anything. The users did not change. The prompts did not change. The routing policy, the cache topology, and the output contract changed.
Enterprise software carried one economic property through every platform shift of the last thirty years: marginal cost per interaction trended toward zero. A database query cost roughly the same whether it returned ten rows or ten thousand. A web request did not scale its cost with the phrasing of the user who made it. Cloud computing changed where the money came from — CapEx to OpEx — but preserved the property. Usage was, to a first approximation, free at the margin.
Generative AI does not preserve it. Every interaction consumes compute in proportion to context length, model size, and output volume, and the relationship is nonlinear. Usage cost is now a function of platform design:
To say that tokens do not correlate to business value is technically true and strategically blind. It is the equivalent of arguing, in the early days of web development, that SQL statements do not correlate to e-commerce revenue. No individual roundtrip carries intrinsic value. The architecture that governs them determines whether the platform is a commercially viable asset or a compounding liability. Database engineers spent two decades internalizing this about query design. The same discipline now applies one layer up, and it has not yet been built.
A two-word prompt does not travel alone. Before it reaches a model, the application layer wraps it: system instructions, safety guardrails, retrieval context, session history. Two words become two thousand tokens. This is the payload the provider bills, and most organizations have never inspected it.
The model processes that payload in two phases with different economics. The first — prefill — loads the entire context at once and is compute-bound. The second — generation — produces the response one token at a time, feeding each token back into the next step. This sequential phase holds hardware longer per token, which is why providers price output at roughly five times input; on Anthropic's published list pricing, the frontier tier runs $15 per million input tokens against $75 per million output.1
Then the compounding effect. If an application maintains conversational memory by resending the full history on every turn, input volume grows with the square of conversation length — that is arithmetic, not an estimate. A twenty-turn session does not cost twenty times a single turn; it costs materially more, and the multiple was set by an architectural decision about state management, not by the user who kept talking.
Nobody decides to pay for this. The architecture decides, and the invoice reports the decision.
Three layers determine token spend. Each is a design decision with a measurable dollar consequence, and each has a natural owner.
The routing layer
Most enterprise AI traffic is low-complexity: classification, extraction, routing, summarization of short documents. An architecture that sends every request to the frontier model pays frontier prices for work a model one tier down performs adequately. The alternative is a multi-model construct: a dispatch layer that matches task complexity to model tier, holding the frontier tier as the exception rather than the default.
The Enterprise Token Economics Calculator models this directly, and the finding from its worked scenarios is worth stating plainly: per-user caps rarely bind, because most users never approach them. In the calculator's professional-services scenario, routing half of frontier-tier traffic to the mid tier accounted for effectively all of a 35 percent reduction in annual cost; the cap contributed nothing because consumption never reached it.3
Caps are ceilings, not levers. Routing is the lever.
The state layer
Natural language defeats exact-match caching. Two employees asking for the same policy in different words are, to a conventional cache, two different requests — and the platform pays full inference price twice. Whether that happens is decided by cache topology: semantic matching upstream of the model, and prompt structure that holds stable content in a fixed prefix so provider-level prompt caching applies. Cached input reads are priced at roughly a tenth of standard input on published rate cards.2 An architecture either captures that discount by design or forfeits it by default. Cache hit rate is a property of the platform, not of the users.
The output contract
Left unconstrained, a model returns prose — framing, hedging, conversational filler — and every word of it is billed at the output premium. An application that needs data can require data: constrained decoding against a schema restricts the model to the fields the downstream system consumes. Given the five-to-one price asymmetry, verbosity is the most expensive default in the stack, and it is a default the platform sets, not one the model imposes.
If cost is decided in these three layers, then cost accountability lives where those layers are designed — and that distributes responsibility in a way most organization charts do not yet reflect.
Platform engineering owns routing policy and cache topology; the dispatch matrix and the hit rate are its numbers. Product owns the output contract; what the application is permitted to return is a product decision with a unit-cost consequence. Finance owns unit economics at the workflow level, not the seat level — the fully loaded cost of one execution of a process, and its variance, because agentic loops introduce variance that per-seat budgeting cannot see. And none of it is enforceable without attribution: user-level, workload-level accounting on every call. Attribution is not itself a cost lever. It is the precondition for every other lever, which is why it comes first.
There is no curriculum to propose here. What each function must become fluent in follows directly from what it owns, and how it builds that fluency is its plan to make. What remains at the executive level is narrower: the map itself, and whether each function knows it owns its territory.
- What fraction of our traffic reaches the frontier tier, and who decided the routing policy — an architect, or the absence of one?
- What is our cache hit rate, and was a target set before the platform went to production?
- What does one execution of our highest-volume workflow cost, fully loaded — and what is the variance across executions?
- When the monthly bill doubles, can we attribute the increase — by user, workload, and layer — within a day?
A leadership team that can answer these four questions has an architecture. One that cannot has an invoice.
The token bill is not a usage report. It is an architecture review, delivered monthly, whether or not anyone reads it that way.