NVIDIA Holoscan Enhances Real-Time Surgical Guidance Through Multi-Modal Imaging

In the quest to enhance surgical precision, NVIDIA Holoscan is making significant strides by fusing multi-modal imaging data to provide real-time guidance during surgeries. This cutting-edge technology is poised to revolutionize surgical procedures by integrating preoperative 3D images with intra-operative video, offering surgeons a comprehensive view that enhances decision-making and surgical outcomes.

Challenges in Multi-Modal Imaging Integration

Surgeons face challenges in seamlessly combining preoperative 3D imaging data, such as CT scans, with real-time intra-operative video. This integration is crucial for accurate navigation and understanding of the patient's anatomy during minimally invasive procedures. However, the lack of seamless access to preoperative data during surgery remains a hurdle.

To address this, NVIDIA Holoscan offers a platform that combines AI, accelerated computing, and advanced visualization to process multi-modal data in real-time. This technology tracks anatomical structures, fuses data, and provides low-latency visualization, crucial for hand-eye coordination and real-time surgical decisions.

ImFusion's Innovative Approach

ImFusion, a member of the NVIDIA Connect Program, has leveraged Holoscan to integrate preoperative data with real-time imaging. Their system creates a digital 3D mesh from preoperative CT scans and overlays it onto live video feeds, allowing surgeons to visualize blended data and make informed decisions in real time.

This integration, built on ImFusion's proprietary SDK, enhances performance and efficiency, enabling the development of next-generation software as a medical device (SaMD). The system's neural networks—stereo depth estimation, optical flow estimation, and segmentation—utilize Holoscan's acceleration capabilities to provide crucial spatial and visual information during surgeries.

Real-Time Performance and Collaboration

Utilizing NVIDIA's IGX Developer Kit with RTX 6000 Ada GPU, ImFusion's system achieves a median frame rate of 13.5 Hz with end-to-end latency below 75ms. This performance is crucial for live surgical applications, providing instantaneous visual feedback and enhancing the surgical scene's view.

The collaboration with ORSI Academy, Europe's largest robotic surgery training center, has been instrumental in strengthening the system's clinical relevance. This partnership aims to improve surgical navigation, particularly in complex procedures, by minimizing damage to healthy tissues.

Future Prospects and Open-Source Contributions

The collaboration between ImFusion and NVIDIA Holoscan is setting new standards in surgical precision, combining AI and accelerated computing for enhanced performance and safety. The contributions to Holoscan's reference applications offer opportunities for medtech customers to integrate and build upon these innovations.

For further exploration and contribution, the Holoscan reference application repository is available, fostering an ecosystem that accelerates the development of AI-enhanced medical devices and advances real-time sensor fusion for surgical guidance.

For more information, visit the NVIDIA Technical Blog.