NVIDIA Blackwell Ultra Delivers 50x Performance and 35x Lower AI Inference Costs

NVIDIA Blackwell Ultra Delivers 50x Performance and 35x Lower AI Inference Costs

The economics of AI inference are shifting fast. NVIDIA's latest Blackwell Ultra platform is delivering dramatic performance improvements that could make AI agents and coding assistants economically viable at scale.

According to new SemiAnalysis InferenceX data, NVIDIA GB300 NVL72 systems now deliver up to 50x higher throughput per megawatt, resulting in 35x lower cost per token compared with the NVIDIA Hopper platform.

What This Means for AI Applications

For developers and businesses building AI-powered products, these improvements directly impact two critical areas:

1. Latency-Sensitive Applications

Agentic AI applications like coding assistants and autonomous agents require low latency to maintain real-time responsiveness across multistep workflows. The Blackwell Ultra platform reduces cost per million tokens by up to 35x at low latency targets, making these use cases more economically viable.

2. Long-Context Workloads

For workloads with 128,000-token inputs and 8,000-token outputs such as AI coding assistants reasoning across entire codebases GB300 NVL72 delivers up to 1.5x lower cost per token compared with GB200 NVL72.

> "As inference moves to the center of AI production, long-context performance and token efficiency become critical. GB300 addresses that challenge directly."

> — Chen Goldberg, Senior Vice President of Engineering at CoreWeave

Who's Already Using It

Leading cloud providers and AI companies have deployed or are deploying NVIDIA GB300 NVL72 in production:

• •Microsoft is deploying GB300 NVL72 for OpenAI workloads via Azure
• •CoreWeave is offering production-ready instances with more than 6x performance gain on DeepSeek R1
• •Oracle Cloud Infrastructure is deploying GB300 NVL72 for supercomputing workloads

Inference providers including Baseten, DeepInfra, Fireworks AI, and Together AI have already reduced cost per token by up to 10x using the earlier Blackwell platform. The Ultra version extends these gains further.

The Bigger Picture

The performance gains come from co-design across chips, system architecture, and software:

• •Higher-performance GPU kernels optimized for efficiency and low latency
• •NVIDIA NVLink Symmetric Memory enabling direct GPU-to-GPU memory access
• •Continuous optimizations from TensorRT-LLM, NVIDIA Dynamo, Mooncake, and SGLang teams

Looking ahead, the NVIDIA Rubin platform (expected later this year) promises another 10x improvement in throughput per megawatt for MoE inference, potentially reducing costs to one-tenth of current levels.

What This Means for You

If you're building AI applications, these cost reductions could make previously impractical use cases viable:

• •Real-time coding assistants that reason across entire codebases
• •Autonomous agents that can run longer workflows without cost concerns
• •Enterprise-scale AI deployments with predictable economics

The trend line is clear: AI inference costs are falling fast, and the platforms enabling these savings are available now.

---

Primary Sources:

• •NVIDIA Blog: Blackwell Ultra Performance Data
• •SemiAnalysis InferenceX Performance Data