NVIDIA details Kubernetes recipe for Vera Rubin NVL72 support
NVIDIA said its AI Cluster Runtime can make Kubernetes clusters ready for Vera Rubin NVL72 compute trays on day one by packaging tuned settings, generating deployment bundles and validating the running cluster. The company framed the effort as part of its DSX AI Factory Platform, and said the work addresses the hardware, driver and scheduler changes required when new GPU systems are introduced into a cloud-native stack.
What changes for Vera Rubin NVL72
In the post, NVIDIA said Vera Rubin NVL72 is an evolution of the same MGX rack-scale architecture used by GB200 NVL72, but with several user-visible changes that platform teams need to plan for. The company said the new system requires the R615.23 driver branch, with early support targeted for R610 and official platform support in R620. It also said Vera Rubin replaces ConnectX-7 NICs and BlueField-3 DPUs with ConnectX-9 SuperNICs and BlueField-4 DPUs, and that DOCA 3.3.0 is the first GA release supporting ConnectX-9.
NVIDIA said the topology also changes in ways that matter for I/O-heavy workloads such as RDMA networking and GPUDirect Storage. According to the post, the Vera Rubin NVL72 compute tray exposes eight ConnectX-9 PCI device groups with 32 functions, compared with four ConnectX-7 adapters on the NVIDIA DGX GB200 NVL72 compute tray. The company also said GPU-to-NIC connectivity is NODE within a socket and SYS across sockets, and that PCI function counts do not represent physical port counts.
The post further said Rubin GPUs introduce new MIG profiles, but that the Vera Rubin recipes described there use whole-GPU allocation and do not configure MIG. It added that multi-node NVIDIA NVLink workloads continue to use IMEX, and that DRA v0.5.0 includes an alpha host-managed option with domain isolation only, while the cluster described in the post uses driverManaged mode.
Validation on early hardware
NVIDIA said it validated the Vera Rubin NVL72 stack end to end using the official GPU Operator v26.7.0 release, the official DRA driver v0.5.0 release and a host-installed R615 driver. The company said both training and inference recipe variants validated successfully for deployment and conformance, and that the training recipe also passed a performance gate based on a cross-node NCCL all-reduce over the NVIDIA NVLink fabric through an IMEX ComputeDomain.
In the validation table, NVIDIA reported a performance result of 744.71 GB/s bus bandwidth at 16 GiB for the first cross-node NCCL all-reduce across both compute trays. The company said the inference variant passed the same deployment and conformance matrix on the identical cluster, but that inference serving performance remains deferred for now.
The post also said the first validation rounds were temporarily blocked by a hardware or fabric issue in one of the early Vera Rubin clusters that kept one compute tray out of the NVLink fabric. NVIDIA said the issue was unrelated to the GPU Operator or the DRA driver and was resolved after the hardware was reseated.
Recipe-based deployment and recorded limits
According to NVIDIA, AICR captures the platform choices required to stand up a working Vera Rubin cluster as a versioned recipe, including driver mode, container-runtime wiring, component versions and platform-specific settings. The company said the new Vera Rubin recipe family for rke2, the vr200 accelerator and Ubuntu ships as Preview, meaning it is intended for early adopters and does not yet carry the production support and lifecycle qualification of a Supported recipe.
The post said the recipe encodes host-managed driver mode because the system image pre-installs the arm64 open-kernel-module driver, while the container toolkit remains operator-managed. It also said the recipe enables IMEX-backed ComputeDomains for the NVL72 fabric, leaves whole-GPU allocation on the device plugin, and pins a set of floors and quirks including Kubernetes 1.34.1 or later, Ubuntu 26.04, and a Vera-optimized arm64 64k-page kernel.
NVIDIA said the first deployment of the bundle targeted a platform combination that had not previously been validated together: arm64, Ubuntu 26.04, a 64k-page kernel, RKE2 and Vera Rubin. The company said that setup moved from cluster handoff to a healthy runtime in hours, and that a second Vera Rubin cluster reproduced the result the following day.
The post also said the validated boundary is not final. NVIDIA noted that inference serving performance still depends on a published runtime image built against CUDA 13.4 or later, and said current public vLLM and ai-dynamo images do not yet meet that requirement in the way needed for a Vera Rubin inference performance floor. The company said the published validation evidence should therefore be treated as the historical record for the recipe coordinate it was captured against.
NVIDIA said the Vera Rubin recipes are published in the open-source NVIDIA/aicr repository in v1.0.0, and that users can generate the validated configuration with aicr recipe commands for either training or inference variants.