Free tool: Stable Houdini puts Stable Diffusion in Houdini
Artist Stanislav Demchenko (stassius) has released Stable Houdini, an open-source toolkit that integrates Stable Diffusion into Houdini.
The toolkit makes it possible to use Stability AI’s AI image-generation model inside Houdini to create concept art, matte paintings or even animations, with the option to use a 3D scene to guide the results.
Stable Houdini, which can be run entirely on a local machine, has been used on projects at virtual production firm EVR Systems, and has just been featured by SideFX on the Houdini product website.
Use Stable Diffusion to generate detailed 2D images matching the composition of a Houdini scene
We’ve covered another free plugin to integrate Stable Diffusion into Houdini on CG Channel in the past: Ubisoft TD Mohsen Tabasi’s StableDiffusion_Houdini.
Like it, Stable Houdini lets you control the generative AI model from inside Houdini, creating 2D images based on text prompts, source images, or the composition of a 3D scene.
The videos above show the toolset in use to generate detailed 2D environment and character images matching simple blocking geometry, and the pose of a stock Mixamo character.
Potential use cases range from creating concept art and storyboard frames to digital matte backgrounds for visual effects shots and even backdrops for a real-time strategy game.
Can be run on an offline machine
Both tools can use open-source Stable Diffusion UI AUTOMATIC1111, making it possible to run the AI model on a local machine, without the need for an official Stability AI DreamStudio account.
AUTOMATIC1111 also includes AI image upscaling models, making it possible to upres the image Stable Diffusion generates to a resolution usable in production.
Can also be used to generate simple animations or to stylise video
On top of that, Stable Houdini can use the open-source ControlNet extension for AUTOMATIC1111 for finer control over Stable Diffusion, particularly when generating multiple frames.
In the video above, you can possible use cases, including creating simple morphing animations, stylising existing video – and even creating an animation of Keanu Reeves dancing based on the Rick Roll video.
The result isn’t particularly temporally stable, but the effect can be quite visually pleasing.
Stable Houdini can also be used for more general image- or video-processing tasks, like generating normal or depth maps matching the current frame.
Licensing and system requirements
Stable Houdini is compatible with Houdini 19.5 and Python 3.9. Downloadable .hda files and a Houdini .hip demo scene are available on GitHub. The source code is available under an Apache 2.0 licence.
AUTOMATIC1111 is available as a compiled binary for Windows 10 and Nvidia GPUs only, though it also runs on AMD and Apple Silicon if you install from source. It is available under at Affero GPL licence.
Download free Houdini Stable Diffusion toolkit Stable Houdini from GitHub
Editor’s note: in the light of Russia’s invasion of the Ukraine, readers may wish to know that EVR Systems is based in Russia.