Foundry releases new AI-powered SmartRoto add-on for Nuke
Foundry has released SmartRoto, a new AI-powered rotoscoping plugin for Nuke, its family of compositing software for visual effects and motion graphics work.
The add-on automatically moves and deforms existing roto splines to follow the subject across shots, making it possible to work “up to 4x faster” compared to manual rotoscoping.
An AI-assisted tool that takes (some of) the drudgery out of roto work
SmartRoto automates the process of moving and deforming roto splines to follow the subject of a shot.
Users manually create roto shapes on the initial frame, then identify further key frames on which there are significant changes to the shot, and the plugin does the rest.
The plugin operates as part of Nuke’s existing Roto node, with “minimal pipeline change or training required”.
Foundry claims that the workflow is “up to 4x faster” than manual rotoscoping in Nuke alone.
So what data was SmartRoto’s AI model trained on?
According to Foundry, the machine learning model on which it is based took four years to create, and was trained on “licensed data appropriate for full commercial use”.
The model itself runs locally, so no user data is shared with Foundry itself.
How does SmartRoto differ from the AI matte generation tools in third-party Nuke plugins?
In its launch material, Foundry is keen to distinguish SmartRoto from automated matting solutions, which also use AI to generate animated mattes to isolate the subject of a shot.
Automated matting is available in a number of third-party add-ons for Nuke itself, including Boris FX’s Silhouette and Mocha Pro, and newer tools like Onyx AI Matte.
Unlike them, SmartRoto uses actual roto splines as inputs, rather than relying on AI image segmentation, using the relationship between splines as context to improve its output.
The workflow does not require a shot to be tracked sequentially frame by frame, with artists able to “jump to any frame and key the shapes”.
Foundry also pitches the fact that SmartRoto workflow is spline-based as making it easy to tweak its output manually.
That will certainly be an advantage over some tools, although both Silhouette and Mocha Pro make it possible to convert mattes into editable splines after they are generated.
How have artists reacted to Foundry’s announcement of SmartRoto?
However, on social media, SmartRoto has drawn criticism for Foundry’s pricing model: a separate $599/year subscription to Nuke itself.
Nuke has a higher-priced edition, NukeX, which adds advanced tools, and a recurring comment on both YouTube and LinkedIn is that SmartRoto should have been one of those tools.
As one senior compositor at Folks VFX put it: “Sorry, this should be part of NukeX. Studios are already struggling to keep the doors open.”
Price and system requirements
Smart Roto is compatible with Nuke, NukeX, Nuke Studio and Nuke Indie 16.1+ on Windows, Linux and macOS. The plugin is rental-only: subscriptions have a standard price of $599/year.
Read more about SmartRoto on the Foundry website
Read more about suggested SmartRoto workflow in the online release notes
Have your say on this story by following CG Channel on Facebook, Instagram and X (formerly Twitter). As well as being able to comment on stories, followers of our social media accounts can see videos we don’t post on the site itself, including making-ofs for the latest VFX movies, animations, games cinematics and motion graphics projects.