Sign up for the newsletter

Signup for the Newsletter

Free tool EbSynth turns video into hand-painted animation

Tuesday, July 9th, 2019 | Posted by Jim Thacker

 
Start-up Secret Weapons has released EbSynth, an interesting free tool for converting video footage into stylised ‘motion paintings’.

The software transfers the look of one or more manually generated style frames to those from the video, creating the look of hand-painted animation.

Based on recent research in style transfer and texture synthesis
EbSynth is based on research from the Czech Technical University in Prague and Adobe Research, led by Secret Weapons founder Ondřej Jamriška, whose work has featured on CG Channel in the past.

The EbSynth website shows the demo video from the team’s most recent paper, Stylizing Video by Example.

Unlike other work done in the field, it isn’t based on machine learning, but uses a “state-of-the-art implementation” of non-parametric texture synthesis algorithms.

Secret Weapons tells us that it now plans to develop EbSynth as a standalone tool.

 

 
Works from source footage and a single hand-painted reference frame
As input, EbSynth only requires the source video – converted to an image sequence in standard 2D file formats – and at least one still keyframe for reference.

The software then automatically transfers the style of the keyframe to the remaining frames of the footage, preserving “texture coherence, contrast and high frequency details”.

Since EbSynth isn’t working from any underlying 3D data, it can’t accurately stylise parts of objects that aren’t shown in the keyframe, or that are missing from it entirely.

That means that the keyframe should match the footage closely – ideally, it should be an overpaint of an actual video frame – and that a new keyframe is recommended after every significant change of perspective.

In addition, the user can supply masks to specify which parts of the video frame the stylisation should be applied to: for example, to stylise a character but not the background.

To judge from the demo video at the top of the story, the results are pretty good out of the box: there are noticeable transitions as the software moves from one keyframe to another, and a ‘halo’ region between character and background when not working with input masks, but the output is very watchable.

Availability and system requirements
EbSynth is available for Windows and macOS. The software is free for commercial use.

Updated: The original version of this story linked to source code for EbSynth on GitHub. Secret Weapons tells us that this is for a previous build and does not match the current implementation.

Download EbSynth from the product website

Tags: , , , , , , , , , , , , , , ,

Comments

You can follow any responses to this entry through the RSS 2.0 feed.

You can leave a response, or trackback from your own site.


© CG Channel Inc. All Rights. Privacy Policy.