DeepSeek-V4 on GB300: 5x Throughput with SGLang

DeepSeek-V4 Performance Breakthrough

You can now achieve a 5x higher throughput with DeepSeek-V4 on NVIDIA's GB300, thanks to a series of coordinated kernel, runtime, and hardening improvements. This advancement maintains the same user interactivity you expect, making it a significant leap for your operations.

The improvements include MHC fusion, token-bucket prewarm, and KV Compression V2, which contribute to stability and efficiency. Your serving pipeline also benefits from better disaggregated decode admission, breakable CUDA graph support, and crucial bug fixes within SGLang and Dynamo.

Technical Advancements

The following features have been added to enhance DeepSeek-V4's performance:

• MHC fusion for improved efficiency
• Token-bucket prewarm for enhanced stability
• KV Compression V2 for reduced latency
• W4A4 MegaMoE for increased throughput
• Stronger SWA budgeting for better resource allocation

These advancements have eradicated instability from the serving frontier, ensuring your operations are robust and efficient.

Quantifying Your Gains

The numbers speak for themselves. With DeepSeek-V4 support live in SGLang since Day-0 (April 2026), these optimizations have drastically improved your efficiency. For instance, the June 2026 MTP curve now delivers approximately 11,200 tokens per second per GPU at roughly 50 tokens per second per user.

What This Means For You

This achievement solidifies the potential of powerful hardware like the NVIDIA GB300 and the Blackwell Ultra. You can now deploy cutting-edge models like DeepSeek-V4 with confidence, knowing that they will be efficient and responsive.

The Bottom Line for Developers

In conclusion, the latest advancements in DeepSeek-V4 have significant implications for your operations. You can now achieve higher throughput, improved efficiency, and increased responsiveness, making it an ideal choice for your AI applications.