NVIDIA’s Vera Rubin NVL72 has made its first appearance in MLPerf Inference v6.1, with the 72-GPU system posting substantially higher inference throughput than an equivalent GB300 NVL72 configuration. The headline result is up to 3.7x higher throughput on Qwen3-VL, while NVIDIA reports up to 2.5x on DeepSeek-R1.
The more useful story for infrastructure buyers is underneath those peak figures. The gains vary sharply by MLPerf scenario; the Rubin submission uses NVIDIA’s highly optimised inference software, and MLCommons does not report measured system power for these results. That makes the benchmark valuable evidence of Rubin’s throughput potential, but not yet proof of lower production cost per token.
NVIDIA’s MLPerf Inference v6.1 announcement describes Vera Rubin as delivering major generational gains over Blackwell. We checked the corresponding public MLPerf entries to see what those gains actually look like across workloads.
What the Vera Rubin MLPerf results actually show
MLCommons lists the submitted system as NVIDIA VR200 NVL72, using 72 VR200 accelerators across 18 nodes. Each node contains four GPUs and two NVIDIA Vera CPUs. The system configuration lists 288GB of HBM4 per accelerator and liquid cooling for the CPUs, GPUs, ConnectX-7 networking and NVLink switches.
| Workload | Scenario | VR200 NVL72 result | GB300 NVL72 result | Approx. ratio |
|---|---|---|---|---|
| DeepSeek-R1 | Interactive | 652,750 tokens/s | 253,506 tokens/s | 2.58x |
| DeepSeek-R1 | Offline | 1,183,327 tokens/s | 679,740 tokens/s | 1.74x |
| DeepSeek-R1 | Server | 1,175,890 tokens/s | 596,944 tokens/s | 1.97x |
| Qwen3-VL-235B-A22B | Interactive | 1,306.6 queries/s | 349.3 queries/s | 3.74x |
| Qwen3-VL-235B-A22B | Offline | 2,392.7 samples/s | 1,305.0 samples/s | 1.83x |
| Qwen3-VL-235B-A22B | Server | 2,323.3 queries/s | 1,210.5 queries/s | 1.92x |
The comparison is useful because both generations are represented by 72-GPU NVL72 systems. It avoids one of the easiest mistakes in MLPerf: comparing total throughput figures from systems of completely different sizes.
The 3.7x headline is real, but it is not a blanket 3.7x upgrade
The largest generational gap appears in Qwen3-VL’s Interactive test. Rubin reaches roughly 3.74 times the result of the 72-GPU GB300 system there. The Offline and Server improvements on the same model are closer to 1.8x and 1.9x.
DeepSeek-R1 shows the same pattern. Interactive throughput rises particularly strongly, while Offline and Server gains are smaller. A buyer modelling a high-concurrency production API therefore should not take the best 3.7x figure and apply it directly to an existing Blackwell bill.
This is one reason MLPerf scenarios deserve more attention than the headline number. Offline inference measures maximum throughput under very different constraints than an interactive workload, where responsiveness must be maintained. The relevant result is the one closest to how the model will actually be served.
Power is the biggest missing number for procurement
The MLCommons system record lists a 2,300W TGP per VR200 GPU. The comparable GB300 entry lists 1,400W per GPU. Across 72 accelerators, simply adding those nameplate figures gives 165.6kW for Rubin versus 100.8kW for GB300.
Those are not measured rack-power figures. They are sums of listed accelerator TGP values. They exclude the distinction between nameplate limits and actual workload consumption and do not capture CPUs, memory, networking, NVLink switches, cooling losses or the rest of the facility.
More importantly, the Rubin results are not marked as MLPerf power submissions. There is therefore no verified system-level energy measurement here that lets a buyer calculate authoritative tokens per joule.
The performance uplift exceeds the increase in listed GPU TGP across the six matched results above, which is encouraging. It still does not establish the production cost advantage. For that, buyers need measured wall power alongside throughput at the latency and accuracy level they intend to run.
A higher MLPerf score does not automatically mean cheaper inference
The number procurement teams ultimately care about is closer to fully loaded cost per useful million tokens. That calculation needs the system’s hourly cost, sustained throughput within the required latency target, utilisation, electricity, cooling, networking, and the cost of keeping spare capacity available for peaks.
Hardware is only one variable. Changing the model can move cost and quality by more than changing the accelerator underneath it, so infrastructure planning should sit alongside an AI model cost and capability comparison rather than treating GPU selection as an isolated decision.
A recurring concern among infrastructure operators evaluating Rubin is also what happens to recently purchased Blackwell capacity. Faster hardware does not instantly make an older fleet uneconomic. If GB300 rental rates fall enough, or the workload does not need Rubin’s latency advantage, older capacity can remain commercially useful. MLPerf cannot answer that depreciation question because it contains no purchase price or Cloud rental-rate data.
Cross-vendor leaderboard comparisons need normalising
MLPerf v6.1 also illustrates why raw leaderboard placement can mislead. A Crusoe submission using 512 AMD MI355X accelerators reaches about 2.41 million tokens per second in the DeepSeek-R1 Server workload, compared with roughly 1.18 million for NVIDIA’s 72-GPU Rubin result.
The AMD system therefore posts the larger aggregate number, but it uses more than seven times as many accelerators. Dividing each result by GPU count would swing the comparison strongly toward Rubin, but that is also an imperfect shortcut because large multi-GPU systems depend on interconnects, memory placement, parallelism strategy, and scaling efficiency.
For an actual purchase, compare systems at the scale you intend to deploy. A one-rack result, a four-rack result, and a 512-GPU cluster answer different capacity-planning questions. Availability and rental economics matter too, particularly for teams deciding between buying hardware and using GPU hosting for AI workloads.
The software stack is part of Rubin’s result
These scores should not be interpreted as a bare-silicon comparison. NVIDIA’s DeepSeek-R1 submission uses its TensorRT-LLM stack, while the Qwen3-VL configuration includes vLLM and NVIDIA Dynamo. Rubin also uses disaggregated serving techniques that separate prefill and decode work.
Precision differs between workloads as well. DeepSeek-R1 is listed with FP4 weights, while the Qwen3-VL submission uses BF16. Those choices affect memory use, compute demand and the optimisation work required to hit the published numbers.
This is easy to overlook in procurement discussions. Buying the accelerator does not guarantee the leaderboard result. Production teams need to establish whether their serving framework, model build, quantisation approach and traffic pattern can reproduce enough of the submitted stack’s performance to justify the hardware premium.
There is no single useful “batch size” to copy from the headline result
The public MLPerf summary exposes the scenario, accelerator count, framework, precision, system topology and hardware notes, but it does not reduce these service-style submissions to one convenient fixed batch-size number.
That is a useful warning for anyone trying to reproduce the result. Server and Interactive inference involve request scheduling and varying concurrency, while Rubin’s serving architecture can separate different phases of generation. A single batch-size figure would hide much of what is driving throughput.
A more useful reproduction request would ask the vendor for concurrency, latency targets, sequence-length assumptions, scheduler configuration, memory utilisation and sustained performance over the traffic profile the buyer actually expects.
Rubin’s preview status still leaves a procurement gap
MLCommons marks the VR200 NVL72 configuration as a preview system. The benchmark therefore gives buyers early independent structure around NVIDIA’s performance claims, but it is not evidence of broad commercial availability, stable rental pricing or mature fleet reliability.
Those questions become important if a team is deciding whether to deploy Blackwell now or defer capacity for Rubin. A theoretical cost-per-token advantage can disappear if the newer system carries a large early-access premium, requires unusually expensive power and cooling infrastructure, or is difficult to obtain at useful scale.
What AI infrastructure buyers should request next?
- Measured wall power: not just accelerator TGP, but power for the complete system while running the target workload.
- A matched production scenario: the same model, quality level, latency target and traffic profile used on both Rubin and the existing fleet.
- Fully loaded pricing: purchase or rental cost plus networking, power, cooling and reserved spare capacity.
- Sustained rather than burst throughput: ideally over a representative production period with realistic request variation.
- Software requirements: exactly which TensorRT-LLM, vLLM, Dynamo, quantisation and serving optimisations are required to approach the benchmark.
- A migration calculation: how much existing Blackwell capacity would actually be displaced, and whether lower Blackwell pricing changes the answer.
DIY AI view: Rubin has a strong benchmark debut, but the cost case needs more data
Vera Rubin NVL72’s first MLPerf appearance provides credible evidence that NVIDIA has delivered a large inference-throughput step over a like-sized GB300 NVL72 system. The most impressive result is Qwen3-VL Interactive at roughly 3.7x the comparable Blackwell result, while several Server and Offline gains sit closer to 1.7x to 2.0x.
The missing piece is economic efficiency. Without measured system power, production pricing and evidence that customers can reproduce the optimised software configuration under their own latency constraints, MLPerf v6.1 cannot tell us what a million useful tokens will cost on Rubin.
For buyers, that should be the next benchmark. Peak throughput establishes Rubin’s potential. Measured cost per useful token will determine whether replacing or expanding a Blackwell fleet makes financial sense.
Reporting basis
Sources
Primary source: NVIDIA Official
