The price of running the same artificial intelligence model can swing from $0.09 to $0.75 per million input words depending on which cloud platform you choose, according to a detailed market analysis published in August 2026. This massive variation exposes a critical blind spot in how companies evaluate AI infrastructure: sticker price alone tells almost nothing about the real cost of deploying machine learning at scale.

The analysis examined 20 concrete, deployable AI inference offerings across six different archetypes, from first-party APIs like OpenAI and Anthropic to specialized inference clouds like Together AI and Groq. The researchers discovered that choosing an inference provider is no longer a simple model-selection exercise. The same underlying model can arrive through a first-party API, a hyperscaler like AWS or Google Cloud, a specialized inference cloud, a routing marketplace, a dedicated endpoint, or a container running inside your own network, and each path exposes different trade-offs.

Why Does the Same Model Cost So Much More on Some Platforms?

The answer lies in what happens after a model processes your request. When researchers tested the open-weight gpt-oss-120b model across different hosts, measured output speed ranged from roughly 92 to 1,916 tokens per second. That's a 20-fold difference in serving performance for identical input. Faster inference means lower latency, which translates to better user experience for interactive applications, but it also means higher infrastructure costs for the provider, which sometimes gets passed along to customers.

Beyond raw speed, the true cost of AI inference depends on factors that token pricing completely ignores. Retries when a request fails, long reasoning traces that generate extra output, tool charges for function calling, cache behavior, actual output length, failure rates, latency penalties, and engineering overhead can reverse a sticker-price ranking entirely. A provider that looks expensive per token might actually save money if it rarely fails and caches results efficiently.

How to Calculate Your True AI Inference Cost

• Measure cost per accepted job: Don't stop at token price. Track the total cost of every request that actually completes successfully, including retries and any failed attempts that consumed credits.
• Account for latency requirements: If your application needs responses in under 500 milliseconds, a cheaper provider that takes 2 seconds is worthless. Factor in whether you need premium shared capacity, provisioned throughput, or dedicated infrastructure.
• Evaluate data retention and privacy controls: "Not used for training," "zero data retention," regional storage, and regional processing are four distinct controls. Some regulated workloads require region-pinned hyperscalers or self-hosted deployment, which changes the economics entirely.
• Compare actual SLA guarantees: Advertised rate limits are ceilings, not reservations. Shared services may throttle or queue requests below a published maximum. A contractual service level agreement or provisioned throughput unit is a fundamentally different product from a high nominal token-per-minute limit.
• Test with your actual workload: Benchmark the providers you're considering with representative requests from your application, not just synthetic tests.

What Are the Six Main Types of AI Inference Providers?

The market has fragmented into distinct categories, each optimized for different use cases:

• First-party proprietary APIs: OpenAI, Anthropic, and Google expose their newest models and native tools first, but you're locked into a single vendor and shared capacity unless you pay for premium tiers.
• Hyperscaler managed services: Azure, AWS Bedrock, and Google Cloud combine model access with enterprise identity, private networking, regional controls, and consolidated billing, making them governance and procurement products as much as inference products.
• Specialized serverless inference clouds: Providers like Together AI, Groq, and Fireworks optimize for speed and can radically change latency for the same model, but they may lack the enterprise features of hyperscalers.
• Routing and aggregation layers: OpenRouter and similar gateways select among upstreams and fail over, buying portability and availability at the cost of another policy boundary and the need to evaluate both the router and every eligible upstream.
• Dedicated and self-hosted deployments: Hugging Face dedicated endpoints and NVIDIA NIM offer isolation and control, but their economics depend on duty cycle, accelerator efficiency, operations, and capacity planning, not token price alone.
• Premium shared-capacity tiers: Services like OpenAI's Fast mode offer SLA guarantees and reserved throughput, appropriate for applications that need predictable performance.

The choice between these archetypes matters more than the choice between models within an archetype. An interactive application might need a fast specialized host to meet latency requirements, while a regulated workload might require a region-pinned hyperscaler or self-hosted deployment regardless of price. A startup shipping across many models might prioritize routing and breadth over the last few milliseconds of latency. A mature workload with steady utilization might find that reserved capacity beats serverless economics even when its sticker price looks higher.

What Specific Pricing Variations Exist Today?

The August 2026 analysis included concrete pricing for 20 offerings. For the gpt-oss-120b model, input costs ranged from $0.09 per million words on Vertex AI to $0.75 per million words on Cerebras and Cloudflare. Output costs ranged from $0.36 per million words on Vertex AI to $0.75 per million words on Cerebras. Together AI and Groq both charged $0.15 per million input words and $0.60 per million output words, matching the mid-market price point.

First-party APIs command premium prices. OpenAI's gpt-5.6-sol model costs $5 per million input words and $30 per million output words on standard shared capacity, or $10 and $60 respectively for its Fast mode with service level agreement guarantees. Anthropic's claude-sonnet-5 was priced at an introductory rate of $2 per million input words and $10 per million output words through August 31, 2026, then rising to $3 and $15 respectively. Google's gemini-3.6-flash offered the lowest first-party price at $1.50 per million input words and $7.50 per million output words.

Hyperscaler pricing depends on region, subscription tier, and whether you're using pay-as-you-go or reserved capacity. AWS Bedrock's pricing for gpt-oss-120b in Sydney was $0.1545 per million input words and $0.618 per million output words, with a 16,000-token maximum output limit. Google Cloud's provisioned capacity model starts at $1,200 per week for one GSU (Google Serving Unit), which represents 675 adjusted tokens per second, with output tokens consuming a 5x multiplier.

Why Privacy and Data Retention Matter More Than You Think

Privacy claims are easy to oversimplify. "Not used for training," "zero data retention," regional storage, and regional processing are four distinct controls that don't always go together. Stateful APIs, abuse monitoring, caches, batch files, grounding, and human-review exceptions can change whether a provider actually meets your privacy requirements. Groq offers zero retention by default, while Together AI doesn't store inputs or outputs by default, but both have exceptions for batch processing. Anthropic's claude-fable-5 model has mandatory 30-day retention and is not eligible for zero-data-retention options.

For regulated workloads in specific jurisdictions, the choice of provider becomes a compliance decision, not a cost decision. Regional controls, residency requirements, and contractual guarantees matter more than token price. This is why hyperscalers remain dominant in enterprise procurement despite higher sticker prices; they offer the governance, identity management, and regional controls that regulated industries require.

The 2026 inference market has matured enough that there is no universal "best" provider. The defensible buying metric is cost per accepted job, not token price. Companies that optimize for sticker price alone while ignoring latency, reliability, privacy, and operational overhead will likely overpay for their actual needs. Those that measure total cost of ownership across all these dimensions will find significant savings by choosing the right provider for their specific workload.