NVIDIA at their COMPUTEX 2023 keynote announces the Avatar Cloud Engine (ACE) – a transformative AI driven foundry service made for game developers.
NVIDIA CEO himself, Jensen Huang, presented first-hand how it could transform your gaming experience by showing-off intelligent interactions with NPCs. NVIDIA partnered up with Convai for a live demonstration on how it works.
The demo is based on three key AI frameworks developed by NVIDIA. They are the NVIDIA NeMo, Riva and Audio2Face. From my understanding, NeMo takes care of the language models. That includes building, customizing and deployment – using proprietary data. Riva on the other hand does speech recognition and text to speech for conversation. Finally, Audio2Face creates facial expressions to match the output from Riva.
The large language models handled by NeMo can be customized with lore and character backstories. NVIDIA also provided countermeasures so gamers would be protected against counterproductive or unsafe conversations via NeMo Guardrails.
According to NVIDIA, game developers can integrate the entire ACE for Games solution or use only the components they need. Developers of middleware, tools and games can use ACE for Games to build and deploy customized speech, conversation and animation AI models baked into their software and games. Audio2Face for example features Omniverse connectors for Unreal Engine 5, so developers can add facial animations directly to characters.
That said, gaming with AI powered NPCs would be more competitive in favor of NVIDIA should studios or developers mutually agree to use such feature as it would require RTX levels of AI or ML processing. Regardless, I cannot wait to see more of this soon.
