NVIDIA Holoscan and RTI Connext Pave the Way for AI-Enabled Medical Devices

The demand for real-time insights and autonomous decision-making is growing across industries, and healthcare and medical devices are no exception. Leveraging real-time edge AI, the next generation of healthcare promises to deliver more precise treatments, improve patient outcomes, and increase operational efficiencies.

Operating rooms of the future will increasingly incorporate AI-enabled and interconnected devices, providing real-time access to holistic patient data, operative insights, decisions, and actions. In such a future, software as a medical device (SaMD) must handle large-scale data with stringent performance and latency constraints while deployed in distributed healthcare systems. This requires interoperability to ensure efficient, reliable, and secure data connectivity between various sensors, displays, controls, and applications without compromising performance or latency.

This post demonstrates how to integrate NVIDIA Holoscan and RTI Connext to create AI-powered medical device applications with high interoperability, low latency, and distributed connectivity. This integration achieves these benefits with minimal overhead and reduced implementation effort and complexity.

NVIDIA Holoscan for Real-Time AI Sensor Processing

NVIDIA Holoscan provides developers with a production-ready framework for building an end-to-end real-time AI sensor processing pipeline, from sensor data ingress to accelerated computing and AI inferencing, real-time visualization, actuation, and data stream egress. This comprehensive solution effectively addresses the multitude of edge AI development challenges, ensuring optimal application performance while abstracting away development complexities, reducing time to market, and offering the convenience of coding in Python and C++.

*Figure 1. NVIDIA Holoscan enables real-time AI for medical devices and much more*

RTI Connext for Real-Time Data-Centric Connectivity

RTI Connext, based on the Data Distribution Service (DDS) standard, streamlines connectivity across complex and scalable systems with a distributed and real-time software communication framework. With Connext, Holoscan applications can integrate with distributed data sources and applications with little overhead while minimizing the implementation effort and complexity of achieving the performance, reliability, and security required by healthcare systems.

Connext provides real-time information exchange between complex system components, enabling stringent reliability, cybersecurity, and performance requirements. Medical systems built on Connext are resilient, self-forming, and self-healing with no single point of failure. The wide range of quality of service tuning options helps meet the need for real-time video and correlated data in distributed, intelligent surgical systems. Built-in security based on the DDS Security standard provides the foundation for authentication and encryption, as well as security logging and granular access control, keeping critical systems safe from security breaches and meeting cybersecurity requirements enforced by regulatory agencies such as the FDA.

*Figure 2. RTI Connext data-centric connectivity enables designing systems around data flow, the most important asset*

Integrating NVIDIA Holoscan and RTI Connext

Today’s healthcare system is built on numerous installed legacy systems that weren’t originally designed with AI capabilities in mind and where NVIDIA Holoscan is not currently natively supported. The integration of Connext with Holoscan enables developers to transform existing legacy installations to AI-enabled and software-defined devices through integration of Holoscan as a sidecar (companion compute module) to those devices where Holoscan is not natively supported.

For example, many existing medical devices are currently Windows-based, particularly in the domain of medical imaging, where Holoscan is not natively supported. Holoscan as a sidecar could bring advanced AI capabilities into robotic surgery systems running real-time operating systems (RTOS) on non-NVIDIA systems. Additionally, low-end sensory medical devices such as patient monitoring could be augmented with powerful AI algorithms while the legacy system’s hardware or software would otherwise limit the addition of such new capabilities.

Holoscan DDS interoperability through RTI Connext DDS offers a solution to these scenarios, providing a scalable, AI-enabled Holoscan sidecar that seamlessly communicates with legacy systems in real time. Holoscan offers exceptional infrastructure for GPU-accelerated SaMD, enabling the innovation and deployment of AI-powered workflows within next-generation healthcare systems, which generally need to operate on large-scale data with very strict latency restrictions.

Using RTI Connext, Holoscan applications can integrate with distributed healthcare systems with little overhead while minimizing the implementation effort and complexity of achieving the performance, reliability, and security required by such systems. In cases where Connext is already being used, introducing new Holoscan-powered AI workflows may be possible without modifications to the existing system.

Example Holoscan Application with RTI Connext Integration

This section provides an example use case: a Holoscan application running on a dedicated system that acts as a sidecar. The application reads frames from a DDS databus using RTI Connext, processes the frame data within a Holoscan workflow, and then publishes the results back to the databus with Connext so the processed frame data can be read by another device for display.

This example enhances a common scenario in healthcare systems in which multiple sensors capture data that is then aggregated for display on a separate monitoring system. Adding AI-powered Holoscan workflows into the middle of this data flow can often be done with little modification to the existing components. Connext helps to bridge these gaps.

The core components of this example are the Holoscan DDS video streaming operators available through nvidia-holoscan/holohub on GitHub. These operators enable Holoscan applications to read and write video frames from a DDS databus in real time. With these operators, Holoscan applications can read video frames from the databus (to be used as the source for workflow processing) and write processed results back to the databus (for consumption by another component).

Combining these two applications demonstrates the sidecar dataflow using three processes:

One dds_video process captures frames from the camera sensor and publishes them to DDS.
A body_pose_estimation process receives the input sensor frames from DDS, processes the frames through the body pose estimation model, then outputs the frames with the inference results overlaid on top of the images to DDS.
Another dds_video process receives the processed frames and renders them to the display.

*Data flow of an example application that uses Holoscan as a sidecar to process video frames that have been published to DDS*

To run this example locally, start by reading the HoloHub DDS Operators documentation regarding the dependency requirements for setting up RTI Connext. To learn how to build and run the applications, see the DDS Support section of the Body Pose Estimation documentation.

Summary

Integrating NVIDIA Holoscan with RTI Connext offers Holoscan developers in the medical device industry numerous advantages in the transition to AI-enhanced systems and devices. These include seamless integration with distributed healthcare systems with minimal overhead, the enhancement of legacy systems with advanced AI algorithms, and more.