Wonder Studio is available
Originally posted on 11 July 2023, and updated with new features: those added since launch are shown in italics. Scroll down for details of new features like Unreal Engine export.
Wonder Dynamics has launched Wonder Studio, its much-hyped AI-powered online platform for inserting 3D characters into video footage.
The platform “automatically animates, lights, and composes CG characters into a live-action scene”, tracking and replacing an actor in the source footage with a 3D character, even matching the facial animation.
It supports custom characters created in Blender and Maya, and can export clean plates, roto masks, and tracking and motion-capture data for use in previs, motion graphics and VFX.
A new AI film-making platform backed by Hollywood’s elite
Wonder Studio got a lot of attention in the mainstream tech press when it was announced earlier this year.
In part, that was due to the names associated with Wonder Dynamics: the firm was co-founded by actor Tye Sheridan, while the advisory board includes Steven Spielberg, who directed Sheridan in Ready Player One.
However, the buzz also came from an impressive demo reel (embedded above), and the fact that Wonder Studio is designed to slot into existing previs and VFX workflows, not to replace them.
According to Wonder Dynamics, the platform’s AI automates “80%-90% of manual VFX work”, particularly repetitive tasks like roto, leaving artists more time to focus on creative decisions.
So how does Wonder Studio work?
Wonder Studio processes single-camera source footage, tracking the motion of an actor, generating roto masks and clean background plates, and replacing the live actor with a CG character.
The lighting on the 3D character is automatically matched to the background plates.
Users can upload up to two minutes of pre-edited video in MP4 or MOV format, with the system automatically detecting cuts, and tracking actors across them.
After identifying the actor to be replaced in a reference frame of video, users can select a 3D character with which to replace them.
Wonder Studio comes with a range of readymade characters, mainly bots and cartoon characters, but you can also upload your own custom characters from Blender or Maya, in .blend or FBX format.
Characters are currently limited to 1.5 million polys, but they can have facial blendshapes and up to 100,000 strands of hair.
Wonder Studio then processes the video, replacing the live actor with the CG character, matching the motion and facial expressions of the character to those of the original actor.
Exports clean plates, roto masks and mocap data as well as final video
As well as simply rendering out the resulting video, which can be done at up to 4K resolution, Wonder Studio can also export the data that was used to generate it.
That includes clean plates and alpha masks for the actor, both exported as a sequence of PNG images; motion-capture data in FBX format; and the camera track.
It is also possible to export the entire scene: initially only to Blender, although subsequent updates have added direct export to Maya and Unreal Engine.
Wonder Studio can also export the scene in USD format, for use in other DCC applications or game engines that support USD, including 3ds Max, Houdini, Omniverse and Unity.
What are Wonder Studio’s technical limitations?
As well as the maximum poly count for 3D characters and the 4K render resolution limit, Wonder Studio has a number of technical limitations.
The biggest is that it doesn’t currently support character-character or character-object interactions, limiting the scope of the shots that it can process.
It also only has partial support for occlusion, so the quality of the output may drop where a character is obscured by foreground objects in the shot.
However, it already looks a promising solution for previs and less demanding VFX projects – and a clear indication of where AI film-making tools could go in future.
Price and system requirments
Wonder Studio is browser-based, and runs in Chrome or Safari. It does not currently support mobile browsers. The platform is compatible with Blender and Maya 2022+.
Lite subscriptions are priced at $29.99/month or $299.88/year, can export rendered video at 1080p resolution, and can export mocap data, clean plates, camera tracks and scene data.
Pro subscriptions are priced at $149.99/month or $1,499.88.year, raise the maximum export resolution of 4K, and also make it possible to export roto masks.
The Terms of Use give Wonder Dynamics a non-exclusive licence to use any content created via the platform, including to develop its AI models.
Read more about Wonder Studio on Wonder Dynamics’ website
Read a list of the latest features in Wonder Studio in the online release notes
See a feature comparison table for paid subscriptions to Wonder Studio
Have your say on this story by following CG Channel on Facebook, Instagram and X (formerly Twitter). As well as being able to comment on stories, followers of our social media accounts can see videos we don’t post on the site itself, including making-ofs for the latest VFX movies, animations, games cinematics and motion graphics projects.