Open-weight model demand is tiny: nearly all DeepSeek traffic fits on a single GPU rack worth $6-7M
Key Points
- Nearly all DeepSeek traffic through OpenRouter fits on a single $6–7 million GPU rack, suggesting open-weight model demand remains a fraction of the infrastructure labs are building.
- Major AI labs are committing gigawatts of new capacity across OpenAI, Anthropic, and others, but observable traffic at model routers reveals a vast gap between available supply and visible consumer inference load.
- The discrepancy likely masks enterprise on-premises deployments and direct cloud provider usage, though the true scale of commercial inference spending remains opaque.
Summary
Open-weight model demand is far smaller than the infrastructure buildout suggests
Nearly all DeepSeek traffic routed through OpenRouter—the largest model router available—fits on a single GPU rack worth $6–7 million, according to Dax at OpenCode. Even if actual demand is 10 times that figure across all inference channels, the total compute requirement still lands below $100 million.
The catch: this figure captures only traffic flowing through OpenRouter and similar routing platforms like Fireworks, Together AI, and Lambda Labs. It excludes enterprises running DeepSeek on their own on-premises infrastructure and direct inference from major cloud providers. Those channels likely account for the bulk of commercial inference spending, though precise figures remain unknowable.
The tension is stark. AI labs are committing to massive new capacity—OpenAI alone is leasing 1.8 gigawatts in Ohio with NVIDIA backstopping 4.25 gigawatts, while Anthropic is locking in 1.7 gigawatts this quarter, including a 360-megawatt facility in Queensland, Australia starting in Q1 2028. Yet the observable open-weight traffic at major routers suggests the actual demand curve for consumer-accessible models remains narrow. The discrepancy points to either hidden enterprise consumption, significant latency in deployment, or a structural gap between available capacity and real inference load that labs are building for future scale rather than present need.
Every deal, every interview. 5 minutes.
TBPN Digest delivers summaries of the latest fundraises, interviews and tech news from TBPN, every weekday.