Check out Wonder Studio’s new Wonder Animation feature
Wonder Dynamics has launched Wonder Animation, a new ‘video to 3D scene’ feature in Wonder Studio, its AI-powered online platform for inserting 3D characters into video footage.
The feature, currently in beta, is intended to “bring artists closer to producing fully animated films” with the platform, turning an edited live-action video sequence into “an animated scene with CG characters in a 3D environment”.
An AI-powered CG film-making platform that can export scenes to Blender, Maya and UE5
Wonder Studio is an AI-powered, cloud-based platform for VFX – and now animation – work.
On its launch last year, it provided an all-in-one solution for inserting CG characters into video footage, tracking the motion of a live actor, replacing them with a CG character, and automatically matching the lighting of the character to the background.
As well as the rendered video, Wonder Studio generates clean plates and alpha masks for the actor, camera tracks, and motion-capture data: it can also be used as a standalone mocap tool.
The data generated can also be exported as a 3D scene to Blender, Maya and Unreal Engine, or in USD format for use in applications like 3ds Max, Houdini or Unity.
New Wonder Animation feature targets 3D animation as well as VFX projects
Part of the biggest update to Wonder Studio since Wonder Dynamics was acquired by Autodesk earlier this year, the new Wonder Animation feature is targeted at 3D animation rather than VFX.
It is designed to work from an edited sequence of live-action shots, which Wonder Studio “essentially [converts] into a continuous 3D scene”.
The platform reconstructs the scene in “a 3D space [matching] the position and movement of each camera [relative] to the characters and environment”.
It works with sequences containing different shot types – wide, medium and close-up – and uses “advanced motion prediction” to reconstruct the movement of characters even where actors are occluded by foreground objects or move partially out of shot “for a few frames”.
Creates a 3D virtual representation of a live-action scene
According to Autodesk’s blog post announcing the new feature, Wonder Animation creates a “virtual representation of [the] live-action scene containing all camera setups and character body and face animation in one 3D scene”.
What it doesn’t do, as far as we can tell, is convert the background of the video to corresponding 3D geometry: users still need to create the 3D environment separately.
We’ve contacted Autodesk to confirm this, and how the 3D scenes Wonder Animation generates differ from those created by the previous scene export system, and will update if we hear back.
Fully editable output rather than a black box solution
In its release materials, Autodesk has been at pains to contrast Wonder Animation with AI video-to-video workflows like those available in Runway’s Gen-3 model.
The toolset is intended to fit into existing workflows, rather than replacing them entirely, and its output is “fully editable”, with users able to adjust the animation, characters, environment, lighting and camera tracking software in 3D applications or game engines.
“There is a current misconception that AI is a one-click solution – but we know that’s not the case,” commented Wonder Dynamics co-founder Nikola Todorovic.
“Unlike the black-box approach of most current generative AI tools on the market, we’re empowering artists to shape their vision instead of just relying on automated outputs.”
Price and system requirments
Wonder Animation is available in beta as part of all Wonder Studio subscriptions. The platform is browser-based, and runs in Chrome or Safari. It does not currently support mobile browsers.
Lite subscriptions have a standard price of $29.99/month or $299.88/year, can export rendered video at 1080p, and can export mocap data, clean plates, camera tracks and 3D scenes.
Pro subscriptions have a standard price of $149.99/month or $1,499.88.year, raise the maximum export resolution of 4K, and also make it possible to export roto masks.
Usage is credit-based: processing one second of video uses 20 credits. Lite subscriptions include 3,000 credits/month, Pro subscriptions 12,000 credits/month.
The Terms of Use give Wonder Dynamics a non-exclusive licence to use any content created via the platform to develop its AI models.
Read an overview of Wonder Animation on Autodesk’s blog
Read more about how Wonder Animation works in the online documentation (Login required)
Read more about the other new features in Wonder Studio in the online changelog
(Login required)
Have your say on this story by following CG Channel on Facebook, Instagram and X (formerly Twitter). As well as being able to comment on stories, followers of our social media accounts can see videos we don’t post on the site itself, including making-ofs for the latest VFX movies, animations, games cinematics and motion graphics projects.