EbSynth 2 can turn video into animation without using AI
Start-up Secret Weapons has launched EbSynth 2, the latest version of its technology for converting video footage into hand-drawn-looking animation.
The browser-based tool transfers the look of a manually edited reference frame to the rest of a video, making it possible to give it the look of hand-painted animation.
As well as complete conversion of video, it can be used for a range of common production tasks for VFX and motion graphics work, including rotoscoping and digital makeup effects.
And interestingly, it does so without using AI to propagate the changes throughout the shot.
What is EbSynth?
We covered the original version of EbSynth on CG Channel a few years ago, back when it was an experimental desktop application.
The underlying technology lets users transform videos by editing a still frame, with EbSynth then propagating the changes automatically to the remaining frames.
Although the result can be a complete transformation of the video into something that looks like hand-drawn animation, EbSynth can also be used for more subtle changes.
For example, it is possible to create digital makeup effects by painting over an actor in one frame of the video, then using EbSynth to propagate the look.
Secret Weapons pitches it as being like Photoshop, but for video.
Does EbSynth use generative AI?
Although technology for stylizing video has become much more common over the past five years, EbSynth isn’t actually a generative AI tool.
Rather than a generative model trained on external data sets, it uses a texture synthesis algorithm.
The approach “preserves the keyframe content at pixel level”, providing more control over the final result.
The tech has been used in production on Richard Linklater’s animated feature Apollo 10½: A Space Age Childhood, and for an animated sequence on Netflix’s Wednesday.
How does EbSynth 2 differ from the original version?
EbSynth 2 takes the core functionality of the original, but presents it in more artist-friendly format.
One change is that it’s now browser-based, rather than a desktop tool, although Secret Weapons still offers studios a self-hosted version that they can install on their own servers.
The new version also has a completely new UI, with familiar features like a timeline, layers and brushes.
However, Secret Weapons tells us that the biggest change is simply the performance of the texture synthesis algorithm, which is now “around 10x” faster, and can run in real time.
That means that workflow is now interactive: users can now preview the results of the changes that they make to a source frame immediately.
Rather than having to prepare complete images in advance, without knowing how they will look when transferred to the video, it is possible to make smaller, iterative changes.
In addition, video tracks and keyframe layers make it possible to work on different elements of shot individually, making workflow a lot faster and more controllable.
Does EbSynth 2 store data online?
EbSynth 2 does have one generative AI feature, but it’s optional.
The new Generate Image option makes it possible to generate the style frame using Stability AI’s SDXL 1.0 model, so if you use it, the resulting video is subject to the SDXL license terms.
If you create the style frame manually, the video is covered by EbSynth’s own terms of use.
Secret Weapons says that videos and still frames uploaded to its server are immediately deleted after EbSynth has synthesized an animation, and the data is not used for AI training.
Price and system requirements
EbSynth 2 is browser-based: the developers recommend Chrome.
A free account provides lets you export video in MP4 format up to 720p resolution.
Pro accounts cost $20/month, and raise the resolution cap to 4K, and add the option to export image sequences in PNG format, and to AI-generate 100 images per month.
EbSynth also offers a command-line version that runs entirely offline, and which is priced on enquiry.
Read more about EbSynth on the product website
Have your say on this story by following CG Channel on Facebook, Instagram and X (formerly Twitter). As well as being able to comment on stories, followers of our social media accounts can see videos we don’t post on the site itself, including making-ofs for the latest VFX movies, animations, games cinematics and motion graphics projects.