NVIDIA Unveils Generative AI Tools for OpenUSD Workflows

NVIDIA has announced the launch of new USD-based generative AI and NVIDIA-accelerated development tools built on NVIDIA Omniverse at SIGGRAPH 2024. These advancements aim to expand the adoption of Universal Scene Description (OpenUSD) across various industries, enabling developers to create highly accurate virtual worlds for the next evolution of AI, according to NVIDIA Technical Blog.

Expanding OpenUSD Adoption

OpenUSD is an open-source framework and data interchange for 3D and simulation workflows. This framework allows developers to build highly accurate, AI-enabled virtual worlds and the tools necessary to create these worlds. The new tools introduced by NVIDIA promise to broaden OpenUSD's applicability to sectors like robotics, industrial design, and engineering.

The new developer tools include:

• NVIDIA NIM microservices and workflow guides for OpenUSD language, materials, spatial intelligence, and physical AI.
• OpenUSD Exchange SDK and USD connectors to support industry-specific data, including robotics and industrial simulation formats.
• NVIDIA Omniverse Kit SDK and developer tooling to build and deploy OpenUSD-based applications from scratch.

Generative AI in OpenUSD Workflows

Generative AI has become relatively straightforward for text- and numeric-based applications. However, applying generative AI to 3D scenes requires both spatial intelligence and physical intelligence. NVIDIA's new NIM USD microservices for OpenUSD language, materials, spatial intelligence, and physical AI bring generative AI to OpenUSD workflows. These microservices allow developers to integrate fine-tuned generative AI copilots and agents into their 3D workflows, expanding USD's adoption into new industries such as manufacturing, automotive, and robotics.

Early adopters of these microservices include WPP, the world's largest marketing and communications company, which is using USD Search and Code to accelerate iteration on creative campaigns at a global scale. Other early adopters include NVIDIA Inception members HOMEE AI and Livvy by Aireal, who plan to use these microservices for intelligent search and AI-powered tools in spatial planning and interior design, respectively.

Building USD-native Applications

Developers can now build OpenUSD-native applications from scratch using the Omniverse Kit SDK and developer tooling. The latest release, Omniverse Kit SDK 106, offers new tooling and enhanced functionality, making development workflows more efficient. This release includes the Omniverse Kit App Template repository, providing a variety of application templates to help developers get started quickly and customize their projects.

The Omniverse Kit App Streaming API, now in early access, allows developers to deploy Kit-based applications and stream interactively from the cloud, enabling seamless integration into web-based solutions.

Connecting Industry-specific Data to OpenUSD

To further advance the OpenUSD ecosystem, NVIDIA announced the upcoming release of the OpenUSD Exchange SDK, available later this year on GitHub. This SDK enables developers to build robust OpenUSD data plugins, converters, and exchangers, facilitating the interchange of industry-specific data with USD. Key formats such as the Unified Robotics Description Format (URDF) are being connected to OpenUSD, enabling roboticists to bring their robot data seamlessly across applications from design to simulation to reinforcement learning.

NVIDIA also announced new developer tools and APIs to stream large-scale OpenUSD scenes to Apple Vision Pro through the NVIDIA Graphics Delivery Network (GDN), currently in early access.

At SIGGRAPH 2024, NVIDIA CEO Jensen Huang participated in fireside chats with Meta founder and CEO Mark Zuckerberg and WIRED Sr. Writer Lauren Goode. These sessions are available on demand through NVIDIA's event page.