Check out free browser-based AI mocap tool Rokoko Vision
Rokoko has launched Rokoko Vision, the new version of its AI-trained browser-based system for extracting motion data from video footage and retargeting it to 3D characters.
The tool – an “evolution” of the firm’s Rokoko Video system – makes it possible to preview and trim motions inside a web browser and export them in FBX format for use in 3D applications like Blender or Maya, or game engines like Unity and Unreal Engine.
The system is free to use with single-camera footage; support for dual-camera footage – the other major new feature in Rokoko Vision – is available with paid subscriptions.
AI-based motion-capture system extracts full-body animation data from video
On its launch last year, the original Rokoko Video became the latest AI motion-capture tool to extract the motion of an actor from video footage and transfer it to a 3D character.
The free browser-based platform enables users to stream footage of an actor from a DSLR camera, webcam or smartphone, or to upload an existing video.
Rokoko then processes the video in the cloud, extracting the motion of the actor’s body and limbs: according to Rokoko’s online FAQs, it takes around 30-40 seconds to process 10 seconds of video.
Now possible to trim animation in your web browser and export it in FBX format
The original Rokoko Video followed a workflow similar to Rokoko’s physical mocap hardware products, like its SmartSuit Pro II inertial mocap suit and Smartgloves, with users cleaning and exporting the animation data using the firm’s Rokoko Studio software.
Rokoko Vision – effectively Rokoko Video 2.0 – makes it possible to do more of the work in the browser, including previewing and trimming motion, and exporting it in FBX format.
However, it’s still possible to use Rokoko Studio to clean up the data before export: the software’s locomotion filter helps to address the foot slippage common with video-based motion-capture systems.
Paid subscriptions to Rokoko Studio also make it possible to export the data in BVH format, or to livestream it to Blender, Cinema 4D, Maya, Unity and Unreal Engine.
Paid subscribers get support for dual-camera set-ups to improve capture accuracy
Another key change in Rokoko Vision is that the system now supports dual-camera set-ups, improving the accuracy of the data extracted.
Rokoko’s website describes dual-camera capture as the “go-to choice for professionals … seeking the utmost in video-to-mocap fidelity and detail”, suggesting that the company is targeting it at final-quality visual effects and game development, as well as for previs and for recording reference animation.
Dual-camera capture follows a straightforward calibration process and supports DSLR cameras, webcams and Android and iOS phones, but it’s only available to users with paid Rokoko Plus and Rokoko Pro subscriptions.
Price and system requirements
Rokoko Vision is free, but you will need to register for a free Starter account on Rokoko’s website to use it, which also makes it possible to use Rokoko Studio.
Rokoko Vision itself is browser-based: although Rokoko has done “limited tests” in Safari, Edge and Firefox, it recommends Chrome. Find details of cameras supported here.
Rokoko Studio is compatible with 64-bit Windows 10+ and macOS 11.0+.
Free Starter accounts support single-camera capture and FBX export; Plus accounts cost $20/month and add dual-camera capture, BVH export, and streaming to DCC apps.
Read more about free browser-based AI mocap tool Rokoko Vision on Rokoko’s website
Read a full list of new features in Rokoko Vision in the online documentation
Have your say on this story by following CG Channel on Facebook, Instagram and X (formerly Twitter). As well as being able to comment on stories, followers of our social media accounts can see videos we don’t post on the site itself, including making-ofs for the latest VFX movies, animations, games cinematics and motion graphics projects.