AI performance costs are falling faster than any previous technology, dropping roughly 13 times per year according to recent research, yet this dramatic price decline masks a more complex reality: newer reasoning models actually consume more computing power per task, even as per-token prices plummet. The gap between what benchmarks show and what's happening under the hood reveals important truths about how AI economics are actually evolving.

How Are AI Costs Actually Falling So Fast?

Epoch AI, a research organization tracking AI trends, found that the price of reaching a fixed performance level on select AI benchmarks has dropped sharply since 2023, falling by about 47 percent per quarter on average. To put this in perspective, if cars depreciated at this rate, a 50,000-euro vehicle would cost less than 70 euros. OpenAI's o3 model provides a concrete example: in early 2025, o3 scored 75 percent on GPQA Diamond, a PhD-level science test, at an estimated 30 cents per question. Eighteen months later, a GPT-5.6 family model hit the same score for four hundredths of a cent, representing 1/725 of the original price.

However, this headline-grabbing figure doesn't tell the whole story. When MIT researchers analyzed the same pricing data using a broader sample of models, they found costs dropping between 5 to 10 times annually, which is still dramatic but considerably lower than Epoch's figures. The difference comes down to what's actually driving the decline.

What's Really Behind the Price Drop?

MIT researchers broke down the cost decline into three distinct factors. When they controlled for cheaper hardware and competitive pricing pressure, they found the actual gain in algorithmic efficiency was about 3 times per year. This means that while the market price is falling 13 times per year, only a fraction of that improvement comes from smarter AI algorithms. The rest comes from cheaper chips and companies competing for market share.

The distinction matters because it reveals something counterintuitive about modern AI development. Epoch's 13x figure includes all market forces, while the MIT team's 3x figure isolates pure algorithmic progress. Both numbers are correct; they're just measuring different things.

• Hardware Improvements: Cheaper processors and more efficient chips account for a significant portion of the price decline, independent of algorithmic breakthroughs.
• Market Competition: Multiple AI companies competing for customers drives down prices faster than technological progress alone would suggest.
• Algorithmic Efficiency: Smarter model designs and training methods contribute roughly 3 times annual improvement when isolated from hardware and competition effects.

Why Better Benchmark Scores Don't Always Mean Better Efficiency?

Here's where the story gets more complicated. MIT researchers discovered that some performance gains simply come from spending more compute on each task. When a new model beats its predecessor on GPQA Diamond, it looks like progress from the outside, but a significant chunk of the improvement comes from using more processing power per question. The model scores higher and costs more to run simultaneously.

This pattern appears across coding and math benchmarks too, though less dramatically. The implication is troubling for anyone relying on benchmark scores to understand real-world efficiency: a single score blends together better training, better data, better architecture, and more test-time compute. You can't tell which factor drove the improvement just by looking at the final number.

Epoch AI tried to guard against this "benchmaxxing" problem, where AI companies optimize specifically for well-known tests without improving real-world performance, by including one test called "Mystery Game Puzzles" based on a game kept secret from developers. Costs drop slowest on that test, which fits the benchmaxxing theory, though it could also reflect the task format or noise in the data.

How Are Companies Actually Using These Cost Savings?

While benchmark costs are plummeting, the economics of running current best-in-class models tell a different story. OpenAI's newer reasoning models like o3 and the recently launched GPT-6 family burn through significantly more compute per task than earlier models, even as the per-token price falls. This creates a paradox: matching last year's top capability is dramatically cheaper, but running today's best model often costs significantly more per query.

OpenAI has responded to this challenge by introducing advanced prompt caching for GPT-6, a technique that reuses computation across multiple requests. The company now gives cache discounts for eligible shared prefixes reused within a 30-minute window, reducing response times and offering developers discounts of up to 90 percent on cached input tokens.

"OpenAI's prompt caching plays a critical role in helping GitHub Copilot deliver fast, efficient experiences at scale. Over the past several months, we've reduced by more than 50 percent the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to our previous baseline," said Mario Rodriguez, Chief Product Officer at OpenAI.

Mario Rodriguez, Chief Product Officer at OpenAI

Real-world deployments show the impact. One company using OpenAI's models improved cache hit rates from roughly 85 percent to consistently above 90 percent in less than a week, further lowering inference costs in production. Another organization moved session agents to explicit cache breakpoints and saw cache hit rates rise from 83 percent to 91 percent, reducing inference costs by 36 percent.

How to Optimize AI Model Costs for Your Application

• Monitor Cache Performance: Use OpenAI's Prompt Caching Dashboard to track hit rates over time and identify unexpected drops in cache efficiency that could indicate optimization opportunities.
• Diagnose Cache Misses: When cache performance drops, use diagnostic tools to compare requests and identify what changed in your model settings, tools, or inputs that prevented reuse.
• Adjust Reasoning Effort Strategically: On GPT-6 models, you can now change reasoning effort between responses without breaking cache, allowing you to use more compute for harder tasks while preserving reusable context for routine follow-ups.
• Stabilize Tool Definitions: Keep tool definitions, schemas, and ordering stable so earlier context stays reusable, using allowed_tools to make only relevant tools callable instead of removing definitions entirely.
• Prewarm Cache During Startup: Prepare known context ahead of time so the model can start responding sooner when a request arrives, moving processing out of the user's wait time.

The broader lesson from both the cost analysis and OpenAI's caching improvements is that raw benchmark prices don't capture the full picture of AI economics. A low-cost model with high latency is useless for a real-time chatbot. A powerful reasoning model might be too slow for automated workflows. A pricier frontier model could still save money if it gets things right more often and cuts down on retries. Choosing the right model requires looking beyond simple price-per-token comparisons to consider quality, latency, context window, and output speed together.

As AI costs continue their dramatic decline, the real competitive advantage will go to companies that understand not just the headline price drops, but the hidden trade-offs between efficiency, capability, and real-world performance.