Get the free MetaHuman Animator Markerless Motion Capture plugin
Epic Games has released the MetaHuman Animator Markerless Motion Capture plugin.
The free Unreal Engine add-on makes it possible to extract full-body animation data, including finger motion, from video of an actor and transfer it to a MetaHuman character inside UE5.
It works with footage shot on a single mono camera, including webcams and mobile phones, making it possible to extract both facial and full-body data from the same footage.
The workflow provides an inexpensive way to generate custom motion-capture data for indie games, animations and VFX projects.
Extends the workflow established by MetaHuman Animator for facial capture
The new plugin extends the workflow established by MetaHuman Animator itself.
Originally released in 2023 as a separate download, but now installed automatically with the Unreal Editor, MHA extracts facial animation from video and transfers it to MetaHumans.
At the time of writing, it works in real time with any mono video camera, including webcams, and with iOS and Android phones via Epic Games’ free Live Link Face app.
In offline mode, it works with depth data captured from a stereo head-mounted camera or on an iOS with a TrueDepth camera (any iPhone or iPad that support Face ID), for greater precision.
Extract full-body and facial solves from the same reference video
To that, the new MetaHuman Animator Markerless Motion Capture plugin makes it possible to extract full-body animation from video captured offline and imported into Unreal Engine
In single-camera mode, both the facial and full-body solves are extracted independently from the same footage.
In dual-camera mode, the two solves are extracted from footage captured using separate cameras: for example, to generate the facial solve from a separate HMC capture session.
The source footage can be captured directly on a camera or phone, or using the iOS version of the Live Link Face app, configured to use a non-TrueDepth camera.
The plugin is based on tech from AI mocap firm Meshcapade, which Epic Games acquired earlier this year, but all processing is done locally, on the user’s machine, not in the cloud.
Social media posts from users suggest that the plugin can process a second of 1080p footage in less than a minute.
Output of the MetaHuman Animator Markerless Motion Capture plugin. The solve was generated from teaching reference footage, which is why some frames are marked up.
‘Surprisingly’ good results in early tests, at least with a static camera
The new plugin was only released yesterday, alongside Unreal Engine 5.8, but feedback from early tests has been encouraging.
The video above shows a test solve generated by Bryan Steagall, Studio Co-head at Area 505 Performance Capture & Animation and founder of mocap systems distributor Kidz Korner Studio.
Writing on LinkedIn, Steagall says that he was surprised by the quality of the results.
“[There are] some floating feet which is very much expected, but I’m even a little surprised that it did so well with the fingers, even at a distance and having them occluded.”
The solve was generated with footage from a static camera: Steagall says that in his tests, the plugin didn’t do as well with footage from moving cameras.
Price and system requirements
The MetaHuman Animator Markerless Motion Capture Plugin is compatible with Unreal Engine 5.8 running on Windows only. It’s free, including for use on commercial projects.
Download Epic Games’ free MetaHuman Markerless Motion Capture plugin for Unreal Engine
Read more about the plugin in the online documentation
Have your say on this story by following CG Channel on Facebook, Instagram and X (formerly Twitter). As well as being able to comment on stories, followers of our social media accounts can see videos we don’t post on the site itself, including making-ofs for the latest VFX movies, animations, games cinematics and motion graphics projects.