Ask what AI inference costs and you will get a list. Every model has its own rate card, usually quoted in dollars per million tokens, often with one price for the text you send and another for the text that comes back. The list changes every few weeks. New models arrive, older ones are repriced, and some are retired.
A list is not a price. To manage a cost, budget for it or hedge it, you need a single number that moves when the cost of what you are buying moves, and only then. For AI inference, that turns out to be harder than it sounds, because the thing being sold keeps getting better.
Two things change at once
Most goods hold still while their price moves. A barrel of a given crude grade is the same barrel this year as last year. A metric ton of copper of a given purity is the same metric ton. When its price rises or falls, the change is a change in price.
AI inference does not hold still. A model released today may answer harder questions, follow longer instructions and make fewer mistakes than one released a year ago, and it may cost less. Price and quality move together, often in the same release.
That creates a measurement problem. Suppose the average price across popular models falls by half over a year. It is tempting to say inference got 50% cheaper. But if the popular models are also much more capable, a buyer who needed only last year's level of capability could have saved far more. And if buyers shifted toward larger, more expensive models because they could now do useful work, the average might even rise while every individual capability got cheaper.
An average of rate cards blends three different things: how much each level of capability costs, which models people choose, and how those models improve. A benchmark that blends them cannot tell you which one moved.
A simple illustration
Imagine a market with two kinds of model. A standard model handles everyday work. An advanced model handles harder tasks. The prices below are invented to show the effect.
- Last year. The standard model costs $4 per million tokens and the advanced model $20. Buyers spend most of their tokens on the standard model, so the average price paid is about $6.
- This year. A new model reaches last year's standard level for $1. A new advanced model, much better than last year's, costs $15. Because it can now do so much more, buyers move a large share of their work to it, and the average price paid rises to about $8.
The average says inference got about a third more expensive. Yet a buyer who needed last year's standard level of capability can now buy it for a quarter of the price, and the most capable work also costs less than before. Both readings are true. They answer different questions, and a benchmark has to be clear about which one it answers.
Holding capability constant
The answer is to fix the thing being priced. Instead of asking what models cost, ask what it costs to buy a defined level of capability: the ability to do a specified kind of work to a specified standard. When a cheaper model reaches that standard, the price of that capability falls. When the models that meet the standard are repriced, the price moves. When a new model exceeds the standard by a wide margin, that improvement does not, on its own, move the price of the standard.
In practice, capability is judged by how models perform on evaluations of the work in question. Levels of capability have to be defined carefully enough that two models placed at the same level really are substitutes for a buyer, and broadly enough that there is a real market of models at each level.
CICI, the Carry Inference Cost Index, is built on this principle. It tracks the price of AI inference at constant capability, so that a move in the index reflects a change in the cost of intelligence itself rather than a change in which models happen to be popular.
A move in a constant-capability index tells you the cost of a fixed level of intelligence has changed. That is the number a buyer can act on.
An old idea in a new market
Statisticians have faced the same problem for decades. A computer bought today at the same price as one bought five years ago is far more powerful. If a price index records no change, it misses something real: the same money buys much more.
Official statistics deal with this through quality adjustment. When a product improves, the improvement is separated from the change in price, so that a faster laptop at an unchanged price counts as a fall in the price of computing. In the United States, a 1996 review of the consumer price index concluded that it had overstated inflation by about a percentage point a year, and that not fully accounting for better and new products was a large part of the reason. Quality adjustment has been a central part of price measurement ever since.
AI inference is an extreme case of the same problem. Quality does not improve every few years; it improves every few months. A measure that ignores it will not just drift, it will tell the wrong story.
Why not follow a single model?
A simpler approach would be to pick one well-known model and follow its price. It has an obvious appeal: one number, from one rate card, easy to check.
It also has three problems. A single model's price is one vendor's decision, which can change for reasons that have nothing to do with the market. Models are retired, often within a year or two, so the series would have to jump to a successor that is both better and differently priced. And buyers rarely depend on one model: they depend on being able to buy a level of capability from whoever offers it at the best price. A single model's price measures a product. A constant-capability index measures the market for what that product does.
Why this matters to the people who pay
Most organizations do not buy a model for its own sake. They buy the ability to get a task done: summarize documents to a certain accuracy, answer customer questions within policy, write and review code to a standard their engineers accept. Their exposure is to the cost of that ability over time.
A constant-capability measure lines up with that exposure in three ways.
- It separates price from progress. When the index falls, the same work has become cheaper to buy. Finance teams can plan around that directly.
- It is continuous through model churn. Models joining or leaving the index do not, by themselves, move its level. The series moves when prices move, or when buyers shift where they spend.
- It can anchor contracts. A price that measures a defined thing over time can be written into budgets, supplier agreements and hedges in a way a shifting average cannot.
What the index does not try to do
A capability benchmark is not a leaderboard. It does not rank vendors, and it does not tell you which model to use. Many buyers will pay more than the constant-capability price because they need a particular vendor, a particular data location or a particular feature, and they should.
It is also not a measure of what the market actually spends. For that, Carry publishes a second index, CRTI, which tracks the realized cost of AI inference in US dollars per million tokens, reflecting the prices and usage the market actually experiences. The two answer different questions. CICI asks what a fixed level of intelligence costs. CRTI asks what buyers are paying. Read together, they show both the changing cost of capability and how quickly buyers move to take advantage of it.
In the illustration above, the two would have moved in opposite directions: the price of last year's capability fell sharply while the average price paid rose. Neither reading is wrong, and a buyer who sees both understands the market better than one who sees either alone.
From measurement to market
Every mature commodity market started with an agreed way to describe the thing being traded and a price people trusted. For intelligence, holding capability constant is that agreed description. It turns a moving list of rate cards into a price series that can be budgeted against, referenced in agreements and, as markets develop around it, hedged.
That is why Carry starts with measurement. A market can only be as good as the price it is built on.
This article is for information only and is not investment advice. CICI and CRTI are in a pre-launch construction phase and published values may be revised. The prices in the illustration are invented and do not describe any real model.