vLLM has made preview support for NVIDIA Vera Rubin NVL72 available. Developers with access to Rubin hardware can use it through nightly containers built with CUDA 13.4 and PyTorch 2.15. The support includes FlashInfer kernels optimized for attention, matrix multiplication and mixture-of-experts computations.
Nightly container supports four model families
The vllm/vllm-openai:cu134-nightly container image is available to developers with access to Rubin. Working with Inferact, NVIDIA and Red Hat, the project has run DeepSeek, Kimi, GLM and MiniMax models on the hardware.
Initial model coverage also relies on compatibility with existing Blackwell kernels. Rubin can run kernels compiled for the forward-compatible sm100f target. Dedicated sm107 kernels are arriving gradually.
MiniMax M3 versus GB200, multimodal Qwen versus GB300
For agentic inference, vLLM published preliminary AgentX results using MiniMax M3. This workload involves long contexts and repeated model calls that put pressure on memory bandwidth and MoE execution. With the same responsiveness target, the project reports up to 7.84× higher throughput per GPU on Vera Rubin than on NVIDIA GB200. At 150 tokens per second, it reports 5.18× higher throughput.
The second comparison uses the multimodal Qwen3-VL-235B-A22B model in MLPerf Inference v6.1 VLM. In vLLM’s preliminary results, the configuration with NVIDIA Dynamo delivered up to 3.7× higher throughput than GB300 NVL72 across offline, server and interactive scenarios. In this tested configuration, Dynamo routes requests to vLLM workers.
The measurements used early hardware and a limited number of nodes. The “up to” figures refer to the best measured configurations, rather than guaranteed gains across all models and workloads.
Expert weights move closer to compute units
FlashInfer 0.7.0 supplies the optimized kernels, covering attention, GEMM matrix multiplication and MoE computations. For MoE decoding, vLLM uses CUDA 13.4 locality domains to place portions of the weights and their associated computations in corresponding regions within the GPU.
Specifically, it splits the FC1 and FC2 expert-weight matrices into column-wise shards and assigns each shard to a locality domain. CUDA Green Contexts then launch a separate kernel for each domain. Compute units read nearby weights instead of repeatedly accessing them across the GPU’s internal memory fabric.
For MiniMax M3’s MoE layers, vLLM reports an average 1.2× speedup in forward passes with small token counts. The result held across TP2, TP4, EP2 and EP4 configurations. TP denotes tensor parallelism and EP denotes expert parallelism; the number specifies the parallel group size.
Options for NVFP4, FP8 attention and expert communication
The CuTe DSL backend for dense NVFP4 matrix multiplication can be selected with --linear-backend flashinfer_cutedsl. It is enabled by default for NVFP4 checkpoints. The separate --moe-backend flashinfer_cutedsl option activates the corresponding NVFP4 MoE kernel.
Setting --kv-cache-dtype fp8 together with the FLASHINFER or FLASHINFER_MLA backend enables the TensorRT-LLM-generation FP8 attention path. The attention backend is selected through --attention-backend.
Expert parallelism is enabled through --enable-expert-parallel together with --data-parallel-size N and --all2all-backend deepep_low_latency. This combination also enables low-latency DeepEP communication and masked grouped GEMM for batched expert formats.
More Rubin kernels are planned
vLLM plans to expand locality-domain support for MoE layers and add a FlashInfer MegaMoE path for sm107. Its roadmap also includes fused mega-kernels intended to reduce kernel-launch and memory overhead.
The project also plans Rubin-specific KDA, MLA, CSA and HCA attention kernels. These are intended for Kimi K3 and DeepSeek-V4.1-Flash.
