NVIDIA cuBLAS Introduces FP64 Algorithms for Faster Scientific Simulations

NVIDIA has unveiled new FP64 algorithms in its CUDA Toolkit 13.4.2, enabling faster and more efficient scientific simulations. The updated cuBLAS library introduces the Ozaki-II and 2M algorithms, which significantly accelerate double-precision general matrix multiplications (GEMM) while maintaining full FP64 accuracy. These advancements are particularly impactful for high-performance computing (HPC) and artificial intelligence (AI) applications that rely on dense matrix operations.

Technical Advancements in cuBLAS

The Ozaki-II algorithm improves upon its predecessor, Ozaki-I, by reducing the number of integer multiplications required for double-precision calculations. Leveraging the Chinese Remainder Theorem (CRT), Ozaki-II performs one integer GEMM per modulus and reconstructs the exact product from residues. This approach achieves linear scaling in GEMM count relative to the number of moduli, compared to the quadratic scaling of Ozaki-I. Additionally, the 2M method optimizes complex matrix multiplications by reducing the required real multiplications from four to two.

Both algorithms utilize NVIDIA's integer Tensor Cores, breaking double-precision numbers into integer components, performing multiplications, and reassembling the results. This approach ensures that the accuracy of native FP64 operations is preserved while benefiting from the speed of lower-precision hardware.

Performance Benchmarks

According to NVIDIA, the new algorithms deliver substantial performance gains:

• DGEMM with Ozaki-II achieves a 5.5x speedup over native FP64 on NVIDIA VR NVL72 GPUs and a 3.9x speedup on NVIDIA GB200 GPUs. It is also 2x faster than Ozaki-I on GB200.
• ZGEMM with the 2M algorithm provides a 5x speedup over native complex FP64 (CFP64) on NVIDIA GB200 GPUs.
• In quantum chemistry simulations using coupled cluster singles and doubles (CCSD), Ozaki-II delivers 2x and 1.4x end-to-end speedups over native FP64 on VR NVL72 and GB200 GPUs, respectively. In mixed-precision mode, the speedups increase to 5x and 3.5x.
• For materials science simulations with BerkeleyGW, ZGEMM using the 2M algorithm achieves 6.6x and 3.9x speedups over native ZGEMM on VR NVL72 and GB200 GPUs, respectively.

Enhanced GPU Capabilities

The introduction of the TI16 numeric format on NVIDIA Rubin GPUs further accelerates Ozaki-I and Ozaki-II computations. This 12-bit integer format, combined with the tcgen05.mma instruction, optimizes performance for GPUs with compute capability 10.7.

Real-World Applications

cuBLAS's advancements have been validated in practical scenarios, including quantum chemistry and materials science. The Automatic Dynamic Precision (ADP) framework ensures that numerical accuracy is preserved, with the library automatically falling back to native FP64 instructions if required. These features make cuBLAS suitable for production deployments without compromising accuracy.

Getting Started

Developers can access the new algorithms by downloading the latest CUDA Toolkit 13.4.2. The cuBLAS documentation provides detailed guidance on API usage, including emulation strategies and mantissa bit control. Sample code is also available on GitHub to help users evaluate performance gains on their workloads.