The unit cost of using artificial intelligence is falling at a speed rarely seen in other technologies. A chart circulated by Andreessen Horowitz, based on data from Goldman Sachs and the U.S. Department of Commerce, compares this decline to that of personal computers: AI has achieved in about three years a price reduction that took PCs nearly two decades to reach. However, corporate bills are not following the same trend.
The key points about AI costs in 20 seconds
- Models are cheaper per token, but applications consume many more tokens.
- Agents plan, consult tools, review results, and repeat steps.
- The same agent-based task can vary up to 30 times in consumption.
- The relevant metric is no longer the API price but the cost per successfully completed job.
This apparent contradiction has a technical explanation. Early chatbots handled relatively simple requests: they received a question, processed the context, and returned an answer. Current agents can make dozens of calls, maintain extensive histories, consult search engines, execute code, open documents, and ask other agents to review results.
Tokens cost less, but each process requires many more.
The cost per million tokens no longer reflects the true expense
Competition among OpenAI, Anthropic, Google, DeepSeek, and open-weight model developers has greatly expanded the price range.
DeepSeek V4 Flash charges $0.14 per million input tokens (no cache) and $0.28 for output tokens. Google places Gemini 2.5 Flash-Lite at $0.10 and $0.40, respectively. At the high end, Claude Fable 5 costs $10 per million input tokens and $50 for output tokens.
OpenAI has also divided GPT-5.6 into three levels. Luna costs $1 for input and $6 for output; Terra, $2.50 and $15; and Sol, $5 and $30. Readings from cache receive a 90% discount, though writing to it is billed at 1.25 times the normal input price.
| Model | Input per million tokens | Output per million tokens |
|---|---|---|
| Gemini 2.5 Flash-Lite | $0.10 | $0.40 |
| DeepSeek V4 Flash | $0.14 | $0.28 |
| GPT-5.6 Luna | $1 | $6 |
| GPT-5.6 Sol | $5 | $30 |
| Claude Fable 5 | $10 | $50 |
This table doesn’t classify models by quality. They differ in capabilities, latencies, limits, data policies, and support levels. A task that requires five attempts with a cheaper model might end up costing more than a correct execution with a more expensive one initially.
This highlights the problem of using per-token price as the sole reference. The comparison ignores how many tokens each model needs to produce a valid result, how often it calls external tools, and how much human labor is required to review its responses.
The most useful metric is the cost per task completed with the required quality. In programming, this could be the cost per issue resolved or per accepted change. In customer support, the cost per conversation closed without human intervention. In document analysis, the cost per case processed correctly.
Agents turn context into a snowball
A traditional chatbot might make one or two calls. An agent follows a longer cycle:
- Interpret the goal.
- Develop a plan.
- Select a tool.
- Read the result.
- Update the context.
- Decide the next step.
- Repeat until completion or reaching a limit.
In each iteration, it can re-process the original question, system instructions, consulted documents, previous responses, and tool results. The context grows—even when the final answer is just a few lines long.
A study by Stanford Digital Economy Lab on programming agents found that these tasks could consume up to 1,000 times more tokens than certain conversation or reasoning exercises about code. It also observed differences of up to 30 times when repeating the same task with the same system. Additionally, spending more tokens didn’t guarantee a higher success rate.
These results apply to programming and shouldn’t be mechanically transferred to all applications. But they do reveal an operational problem: agents follow variable trajectories and can’t always predict how much a task will ultimately cost.
Calling external tools adds further costs. Web searches can generate multiple billable queries. Running code produces results that are fed back into the model as new input tokens. Screenshots, PDFs, and images must also be converted into representations that consume context.
Google charges separately for Search and Maps integrations, while OpenAI notes that some tools apply per-call fees in addition to model consumption. Very long contexts can also be more expensive: in GPT-5.6, requests exceeding 272,000 tokens double the input fee and increase the output fee by 50%.
Why spending increases even when AI is cheaper
Falling prices are expanding the number of processes worth automating. A company that previously used AI to summarize some documents might now apply it to all emails, contracts, incidents, code repositories, and meetings.
Cost savings per unit drive higher demand. Andreessen Horowitz argues that the reduction in AI prices is accompanied by an even greater increase in consumption—similar to the Jevons paradox: improving resource efficiency can raise overall utilization.
There’s also a shift from point responses to prolonged work. GPT-5.6 allows coordination of sub-agents in parallel. Claude Fable 5 is designed for projects that can stay active for days. These capabilities boost productivity but also multiply inference, tool use, and context processing.
Therefore, a 50% reduction in token price doesn’t necessarily mean that the total budget will halve. If the new application makes ten times more calls, overall spending will still grow.
Optimization should address the entire workflow:
| Technical measure | Expected effect |
|---|---|
| Routing between models | Reserve expensive models for difficult tasks |
| Step limits and retries | Prevent infinite agent loops |
| Prompt and document caching | Reduce repetitive processing |
| History summarization | Prevent indefinite context growth |
| Batch processing | Lower costs when immediate response isn’t needed |
| Structured results | Avoid long, hard-to-validate responses |
| Automatic evaluations | Identify which model performs best per case |
| Task observability | Relate consumption, latency, errors, and quality |
The cheapest option doesn’t always have to be a smaller model. A more capable model may finish faster, use fewer tools, and need less correction. Decisions should be based on testing with real product tasks, not only on listed prices.
A multi-model architecture reduces costs and dependency
Rapid price variation is another reason to avoid building an application tightly coupled to a single API.
An abstraction layer can normalize messages, tools, structured responses, metrics, and retry policies. Above it remains the product logic. Below, models can change based on cost, quality, latency, data location, or availability.
This architecture enables using a cost-effective model for request classification, a specialized one for coding, and a frontier model only when the system detects a complex exception. It also facilitates integrating open-weight models within the same infrastructure when data must stay within a controlled environment.
Portability isn’t automatic. Each provider uses different functions for tools, caching, reasoning, structured outputs, and context. An application heavily dependent on an exclusive feature will face migration costs despite a compatible interface.
Therefore, testing model exchangeability before it’s needed is essential. Teams can maintain evaluation sets, record reference responses, and periodically measure costs to achieve the same quality across providers.
AI cost remains an attractive metric but will become less decisive alone. The technical advantage lies in controlling how much AI the system consumes to solve each problem.
Frequently Asked Questions
Why does the bill increase if models are cheaper?
Because agent-based applications make more calls, accumulate larger contexts, and use external tools. Volume can grow faster than the reduction in per-token price.
What is the best metric to measure AI application cost?
The cost per correctly completed task. It should be combined with success rate, latency, retries, and human review time.
Does always using the cheapest model reduce spending?
Not necessarily. A cheaper model might consume more tokens, fail more often, or require multiple attempts. Costs should be measured over the entire workflow.
What does a multi-model architecture offer?
It allows assigning each task to the most suitable model, introduces alternatives in case of failures or price changes, and reduces reliance on a single provider.
