Stability.ai releases its Stable Diffusion add-on for Blender
Stability.ai has released Stability for Blender, a free add-on that integrates Stable Diffusion into Blender.
The tool makes it possible to use the open-source AI image generation model inside Blender, either to convert existing images to textures, or to use a 3D scene to guide the image generated.
It’s even possible render out entire Blender animations while keyframing the Stable Diffusion parameters, to create interesting stylised motion-graphics-style effects.
Convert simple 3D geometry into illustrations or concept art in a wide range of visual styles
There are aleady free plugins integating Stable Diffusion into Blender – we covered Ben Rugg’s AI Render last year – but Stability for Blender is Stability.ai’s official version.
Like AI Render, it links Stable Diffusion into Blender – at its simplest, making it possible to enter the text prompt inside Blender, rather than using a web interface.
However, a much more interesting way to use the plugin is to use a 3D scene in Blender to guide the image that Stable Diffusion generates, in addition to the text prompt itself.
Blender renders the scene, then uses that render as an additional prompt for Stable Diffusion, making it possible to generate images matching the composition of the 3D scene.
The workflow turns Stable Diffusion into something akin to an AI render engine.
With it, users can quickly exploring variant looks for the scene – or even using simple blocking geometry to generate detailed concept art or illustrations, choosing a range of preset styles from within the UI.
Can even stylised rendered animations
In addition, Stability for Blender can convert simple guide images into diffuse textures within Blender, or even stylise entire animations.
Users can render existing Blender animations while keyframing the Stable Diffusion control parameters, which results in some interesting motion-graphics style effects like those shown in the image above.
As ever, before using Stable Diffusion, it’s worth reading up about its strengths and limitations: The Verge has a good summary of the wider copyright and ethical issues posed by AI art tools.
Price, licensing and system requirements
Stability for Blender is compatible with Blender 3.0+. The source code is available under a MIT licence.
The plugin itself is a free download, but to generate images using it, you will need to link it the API key of an account at DreamStudio, Stability.ai’s browser-based frontend.
Registering for an account is free and comes with 100 credits for generating images: the number needed depends on the resolution and number of processing steps used. Further credits cost $10/thousand.
Read more about Stability for Blender in the online documentation
Download Stability for Blender from Blender Market or from the plugin’s GitHub repository