Free tool: Mohsen Tabasi’s StableDiffusion_Houdini 2.0
Ubisoft Singapore senior technical artist Mohsen Tabasi has released StableDiffusion_Houdini 2.0, the latest version of his free tool that links Stable Diffusion into Blender.
The plugin makes it possible to use the open-source AI image-generation model inside Houdini, in order to use 3D scenes to guide the image that Stable Diffusion generates.
With it, users can convert renders of simple blocking geometry into detailed illustrations and concept art.
Version 1.0 required an online Stability AI DreamStudio account to generate images, but version 2.0 makes it possible to set up a server based on open-source tool AUTOMATIC1111 and run entirely on a local machine.
StableDiffusion_Houdini being used to generate a 2D image in the style of baroque painter Nicolas Poussin matching the composition of simple 3D blocking geometry inside Houdini.
Convert simple 3D geometry into illustrations or concept art in a wide range of visual styles
Open-sourced last year, and part of a new wave of AI art tools, Stability AI’s Stable Diffusion model generates 2D images based on user’s text prompts and, optionally, guide images.
StableDiffusion_Houdini makes it possble to use a 3D scene to generate that guide image, using Houdini’s Procedural Dependency Graph (PDG) to submit a native Houdini render of the scene as the init_image.
That makes it possible to use simple blocking geometry to generate detailed concept art or illustrations.
The 3D scene largely dictates the composition of the image, to which Stable Diffusion transfers a new visual style – something that can be guided by entering text prompts within Houdini.
The initial image and the Stable Diffusion output are both shown in a Houdini compositing viewer, and you can also adjust Stable Diffusion parameters within Houdini to adjust the look of the final image.
Can now run entirely on a local machine
On its original release last year, StableDiffusion_Houdini required you to connect an official Stability AI DreamStudio account in order to generate images.
Signing up for an account is free, but generating images consumes credits, and DreamStudio generates images under a CC0 licence, which limits their use in commercial projects.
However, StableDiffusion_Houdini 2.0 adds the option to install a server running open-source Stable Diffusion UI AUTOMATIC1111 on your PC, and connect to that local server.
(AUTOMATIC1111 also includes a number of useful extras including AI-based image upscaling and face restoration tools, although we aren’t sure whether you can control them directly from within Houdini.)
Further reading and related tools
As ever, before using Stable Diffusion, it’s worth reading up about its strengths and limitations: The Verge has a good summary of the wider copyright and ethical issues posed by AI art tools.
If you’re looking to use Stable Diffusion with other DCC applications, we’ve previously covered AI Render, a free add-on that integrates Stable Diffusion into Blender in a similiar way.
Licensing and system requirements
StableDiffusion_Houdini 2.0 is available as a Houdini Indie .hdalc file under a MIT licence.
AUTOMATIC1111 is available as a compiled binary for Windows 10 and Nvidia GPUs only, though it also runs on AMD and Apple Silicon if you install from source. It is available under at Affero GPL licence.
Download StableDiffusion_Houdini from Mohsen Tabasi’s GitHub repository
Find instructions for using StableDiffusion_Houdini with AUTOMATIC1111 in the product wiki