At Unreal Fest Seattle 2024, NVIDIA has introduced a suite of powerful new plugins designed for Unreal Engine 5. These updates, part of the NVIDIA ACE (Avatar Cloud Engine) suite, aim to make it easier for developers to create and deploy AI-driven MetaHuman characters on Windows PCs.
New ACE Plugins for Unreal Engine 5:
- Audio2Face-3D Plugin: This plugin leverages AI to generate realistic lip sync and facial animations directly within Unreal Engine 5, eliminating the need for complex manual animation work.
- Nemotron-Mini 4B Instruct Plugin: Designed for interactive character dialogues, this plugin generates AI-driven responses, making conversations with digital characters feel more natural and immersive.
- Retrieval Augmented Generation (RAG) Plugin: This tool enhances character interactions by feeding contextual information into conversations, improving the relevance and depth of dialogue.
Audio2Face-3D Plugin for Autodesk Maya
In addition to its Unreal Engine integration, the Audio2Face-3D plugin is now available for Autodesk Maya, enabling high-quality facial animations based on audio input. With source code available for customization, it facilitates seamless integration into a wide range of digital content creation workflows, bridging the gap between Maya and Unreal Engine 5.
Unreal Engine 5 Renderer Microservice with Pixel Streaming
This new feature takes advantage of Epic Games’ Pixel Streaming technology, now enhanced with support for NVIDIA’s ACE Animation Graph microservice and Linux OS in early access. Developers can stream high-quality, AI-driven MetaHuman characters to any device through WebRTC (Web Real-Time Communication), offering cross-platform capabilities with impressive fidelity.
Benefits for Developers:
- Streamlined AI Integration: Developers can easily incorporate AI-powered digital humans into their projects, whether for games, simulations, or other interactive applications.
- Optimized Microservices: The plugins are optimized for low latency and minimal memory usage, making them efficient to deploy on Windows PCs.
- Scalability: Designed with cloud deployment in mind, these tools offer scalable solutions for creating and managing digital humans across various platforms.
- Customization: With open access to source code, developers can tailor these tools to fit their specific project requirements.
Developers interested in these innovations can apply for early access to download the Unreal Engine 5 renderer microservice, which includes support for the Animation Graph microservice and Linux OS. Meanwhile, the Maya ACE plugin is already available on GitHub for those looking to integrate it into their workflows.
