For most of the AI buildout, "compute" meant training. A lab raises money, rents ten thousand GPUs for a few weeks, burns a nine-figure sum, ships a model. The GPU market optimised around that: long contracts, dense interconnect, guaranteed capacity. Every pricing conversation started from the assumption that the buyer was a well-capitalised lab doing a discrete, finite, enormous job.

2026 is the year that assumption flipped.

The crossover, in numbers

Multiple independent trackers now agree on the direction, even if the exact figure wobbles:

  • Deloitte's 2026 TMT Predictions put inference at roughly two-thirds of all AI compute this year — the first time it has exceeded training.
  • Gartner forecasts 2026 global inference spending at about $23.3 billion, overtaking training spend of roughly $19 billion, or 55% of total AI IaaS spend. It projects that share rising to 59% in 2027.
  • IDC tracks the same curve from the workload side: inference was about 65% of AI server workloads in 2024, heading to roughly 70% in 2026 and 73% by 2028.

Three different methodologies, three different numbers, one conclusion. The careful version: inference is now the majority of the load, and it's growing faster than training. Anyone telling you the precise ratio to a decimal point is over-claiming, because nobody measures the same thing — but nobody credible disputes the crossover happened.

Why the shape matters more than the size

A training run is a spike. It's huge, it's predictable, it has a beginning and an end, and you can plan capacity around it months in advance. An inference fleet is a horizon. It runs every hour of every day, it scales with users, and its worst enemy is idle capacity — because a serving cluster that isn't serving is burning power and depreciation for nothing.

That distinction was always true for classic "chatbot" inference. What changed in 2026 is that inference stopped being a single cheap call and became a chain.

Here's the mechanism, and it's the most under-discussed shift in the industry. A traditional inference request is one forward pass: user asks, model answers, done. A reasoning model, or an agentic workflow, does something structurally different — it plans, calls tools, checks its own work, revises, and iterates before it produces an answer. That is not one forward pass. It's dozens, sometimes hundreds, chained together, each one consuming context.

Nvidia's own framing at GTC 2026 was blunt: with "thinking" AI, "the thinking happens before the answer." The company's Jensen Huang argued the result is inference demand that can scale by orders of magnitude, because the same request now costs far more compute than it did a year ago. That's a vendor talking its book — but the unit economics back it up. Agent-driven applications have shown token processing volumes rising as much as tenfold in the same period, precisely because when the cost per call drops, developers build agents that make far more calls.

The paradox: prices collapse while the bill grows

Here's the part that confuses people watching from the outside. If inference is exploding, why is the price of a token falling off a cliff?

Both things are true, and the explanation is the whole story of 2026. Enterprise inference pricing fell to roughly $1.16–$1.18 per million tokens by August 2026, down 43% in ten weeks and about 88% from a 2023 baseline. Open-weight models — many from Chinese labs — now carry ~61% of top-model token traffic on router platforms, at roughly $0.83 per million tokens against $6.03 for proprietary alternatives. The floor has collapsed.

At the same time, the ceiling went up. Frontier labs launched their most capable agentic models at around $10 per million input tokens and $50 per million output — effectively doubling the cost of the previous generation. The market bifurcated: cheap commodity intelligence at the bottom, expensive autonomous capability at the top, and a wide, widening gap between them.

So the total bill grows for two reasons at once. Volume explodes because the commodity tier is nearly free. And the premium tier gets more expensive per unit because it's doing qualitatively more work. Falling prices didn't shrink the compute market. They grew it, the way cheap bandwidth grew the internet.

What this does to where compute gets bought

This is where the story stops being about economics and starts being about infrastructure strategy. Three consequences follow directly:

1. Elasticity becomes the most valuable property of a cluster. Training wants dense, tightly-coupled, guaranteed capacity. Inference wants the opposite: capacity that can expand on a Tuesday and contract on a Wednesday, without a three-year commit. Historically you paid a luxury premium for that flexibility from hyperscalers. As inference becomes the dominant load, the premium stops being a luxury and starts being a tax on your core business.

2. The value migrates from the chip to the scheduling. If the bottleneck is keeping thousands of heterogeneous nodes busy against spiky, latency-sensitive demand, then the moat isn't the GPU — it's the orchestrator that can place work, checkpoint it, and recover it when a node disappears. That's an engineering problem, not a capex problem, which is exactly why smaller players have a shot here.

3. The reliability objection narrows but doesn't vanish. The honest case against decentralised compute has always been: nodes vanish mid-job, verification is hard, and you can't trust a garage rack with a production workload. That's still true for a frontier training run. But for inference it matters far less — an inference request is short, stateless, and retryable. If a node dies, you resend the request. The failure mode that makes decentralised capacity unsuitable for training is almost irrelevant for serving. This is the single most important structural fact in the sector: the workload that's growing fastest is the one decentralised networks are best suited to serve.

The catch nobody should skip

None of this means decentralised compute wins inference by default. Two hard problems remain unglamorous and unsolved at scale.

Verification. If you can't confirm that a node actually ran your job on the hardware it claimed — and ran it correctly — then cheap capacity is worthless for anything you'd be embarrassed to get wrong. The networks making real progress here are the ones investing in attestation and cryptographic verification rather than marketing. This is the fight that decides the middle of the market.

Latency and locality. Inference for interactive products is latency-bound. A node on the other side of the world is cheap and useless if your users feel the round trip. Decentralised networks have to solve placement, not just price, to serve real-time workloads.

Both problems are engineering problems with known shapes. Neither is a law of nature.

What to actually watch

Ignore the headline ratio — it's already decided. Watch four things instead:

  • The commodity-to-frontier price gap. If it keeps widening, routing becomes the whole game and the "average token price" becomes meaningless.
  • Verified inference benchmarks. The first network to publish credible, third-party-auditable proof that a job ran on claimed hardware at claimed quality will set the bar everyone else has to clear.
  • Serving latency from decentralised pools. If it converges on centralised latency for common models, the price difference stops being a trade-off and starts being free money.
  • Power, again. Inference runs continuously, so it competes for baseload power, not peak capacity. The grid, not the GPU, may be the binding constraint by 2028.

The training era asked "who can afford the biggest spike?" The inference era asks a different question: "who can keep the most machines busy, cheaply, for years?" Those reward different infrastructure — and the industry is only starting to price that in.