Omniverse Machinima and Audio2Face to enter beta
Omniverse Machinima will enable users to create animation using game assets, as shown in this making-of for Nvidia’s original trailer for the app, which is due to launch in open beta at the end of GTC 2021.
Omniverse Audio2Face, a companion tool for generating facial animation automatically from audio sources, is also due to become available in open beta at the end of the week.
Both will be available as apps within Omniverse, Nvidia’s USD-based online collaboration platform, itself currently in free open beta.
Create cinematic-quality animations using stock props and characters from videogames
Originally announced last fall, Omninverse Machinima will enable users to create animation using existing assets from games, using a set of accompanying AI-driven animation tools.
Users will also be able to import stock content from online marketplaces; or custom assets created in other DCC applications, using Omniverse’s growing library of connection plugins for DCC applications.
The assets can then be assembled into background environments within Omniverse: the video above shows a brush-based layout workflow, with a user painting trees over terrain.
Audio2Face converts audio recordings of dialogue into lip-sync and facial animation in real time
For character animation, Nvidia provides a set of AI-trained add-on tools for generating believable human motion from audio or video recordings.
They include Omniverse Audio2Face, Nvidia’s own app for converting audio into facial animation.
As well as processing offline recordings of dialogue, it will be possible to connect a microphone and perform live, driving a 3D character’s facial animation in real time.
Users can “edit various post-processing parameters” to adjust the resulting performance, with support for direct blendshape editing due “at a later date”.
Audio2Face can also retarget facial animation from one character to another.
Full-body pose estimation tools developed by AI specialist wrnch
For full-body animation, Nvidia has partnered with AI specialist wrnch.
Add effects and render out using the Omniverse platform’s built-in tools
Once character work is complete, Machinima users can add effects and export the final cinematic animation using the other components of the Omniverse platform.
The finished animation can then be rendered via Omniverse RTX Renderer, the platform’s GPU-based real-time ray tracing render engine.
Pricing and availability
Omniverse Machinima and Omniverse Audio2Face are due out in open beta later this week, at the end of GTC 2021. Nvidia hasn’t announced a final release date for either app.
Both will be available for Windows 10 only, and require a compatible Nvidia GPU: at least a GeForce RTX 3060 or Quadro RTX 4000 in the case of Machinima; any RTX card in the case of Audio2Face.
The tools are provided as apps within Omniverse, so to use them, you will need to install Omniverse first, then install them via the Apps section of the Omniverse Launcher.