NVIDIA adds memory-enforced GPU sharing to CUDA 13.4

NVIDIA said its CUDA 13.4 release adds transparent memory enforcement for shared GPU workloads, a change aimed at letting Kubernetes clusters divide GPU capacity more precisely than time-slicing alone. In a technical blog published Oct. 27, 2026, NVIDIA said the new capability is designed to keep workloads within assigned memory boundaries while preserving shared access to a GPU.

What CUDA 13.4 changes

According to NVIDIA, CUDA already supported several GPU-sharing approaches, including Multi-Instance GPU, which splits a GPU into hardware-isolated instances, and Multi-Process Service, which lets processes run concurrently in separate address spaces. The company said CUDA 13.4 adds a memory boundary that is transparent to the application in a new time-sharing mode, alongside the compute-sharing mode already available in MPS.

NVIDIA described four related behaviors in the release: a workload can request an exact amount of memory, in which case an allocation above that size fails; it can request a range with a guaranteed minimum and a higher ceiling when memory is available; it can choose between time-sharing and sm-sharing compute modes; and it can see only a fraction of total GPU memory through a transparent memory limit.

How Kubernetes enforcement works

NVIDIA said CUDA provides the memory enforcement capability, but Kubernetes needs node-level software to apply it. The blog says a node must supervise the daemons the feature depends on, apply each Pod container’s memory limits and compute mode before the process starts, and report usage per fraction rather than per whole GPU.

To do that, NVIDIA introduced an open-source Kubernetes operator called gpu-fractioning under the KAI Scheduler organization on GitHub. The operator runs alongside the NVIDIA GPU Operator and handles three tasks: setting up and running MPS on each GPU node, applying workload memory limits and compute mode, and measuring memory and compute use per fraction rather than per GPU.

NVIDIA said schedulers pass GPU allocations to the operator through per-container Pod annotations. The annotations identify guaranteed GPU memory, a maximum memory limit, the compute mode, and the assigned GPU devices. The company said any Kubernetes scheduler that can write those annotations can enforce GPU fractions with CUDA, though the scheduler still has to choose a GPU with enough free memory and avoid mixing compute modes on the same GPU.

KAI Scheduler integration

NVIDIA said KAI Scheduler is the first scheduler integrated with gpu-fractioning. Starting with v0.18.0, KAI Scheduler includes a GPU-sharing mode called NvFractions, which writes the annotations and relies on gpu-fractioning to enforce them on the node. NVIDIA said the other sharing modes in KAI Scheduler remain available and that administrators choose one per cluster.

The blog also outlined two ways to request memory: as an absolute size using NVIDIA’s per-container annotation and standard Kubernetes units, or as a portion of a GPU using KAI Scheduler’s existing gpu-fraction annotation. Limits are optional, NVIDIA said; if omitted, the limit matches the request. For portion-based requests, the limit must be greater than the requested fraction and less than 1.

Time-sharing versus sm-sharing

NVIDIA said compute can now be divided in two explicit modes. Time-sharing is the default and serializes workloads, with each taking turns on the GPU and running in its own error domain. SM-sharing runs workloads in parallel by splitting streaming multiprocessors among them, with hardware allocating and releasing SMs as needed; NVIDIA said this mode places workloads in a single error domain.

The company said a shared GPU can run only one compute mode at a time, so KAI Scheduler will not place time-sharing and sm-sharing workloads on the same GPU. It said the scheduler tracks this per GPU, not per node, so one node can host both modes on different GPUs.

Example use cases and timing

NVIDIA gave examples of a multi-GPU inference service requesting 40Gi of memory per GPU with room to grow to 60Gi and using sm-sharing, as well as a development or batch workload requesting half a GPU on two devices with the option to grow to 80% under time-slicing. The company said the first example suits workloads that are trusted to run together, while the second gives each workload its own error domain.

NVIDIA said the open-source GPU Fractioning Operator and KAI Scheduler are available under the KAI Scheduler organization on GitHub, with installation and configuration instructions for all three sharing mechanisms. The company also pointed to its KubeCon + CloudNativeCon NA 2026 appearance in Salt Lake City, Utah, Nov. 9-12, as a place to discuss the tools.