Unity unveils new generative AI service Unity Muse
Unity has unveiled Unity Muse, a new generative AI service that will enable games artists to create assets based on a combination of text prompts and rough sketches.
The platform was announced alongside Unity Sentis, which will enable game developers to embed AI models into the Unity runtime itself. Both are currently in invite-only beta.
Unity Muse: generate games art assets from text prompts and sketches
The new services are the first public results of the Unity AI initiative, announced at GDC 2023 via a memorably vague teaser campaign.
Of the two, generative AI platform Unity Muse is the one aimed at games artists.
Unity’s blog post describes it as enabling users “to create almost anything in the Unity Editor using natural input such as text prompts and sketches”.
Currently limited to AI-based natural language searches
The initial release is rather more limited, since it consists of only the AI search component of the platform.
With Muse Chat, users can type in natural-language queries to have Muse Chat search Unity documentation and training to get “well-structured, accurate, and up-to-date information … including working code samples”.
Similar functionality is currently available in third-party add-ons like AICommand, the proof-of-concept integration of ChatGPT into Unity that Unity engineer Keijiro Takahashi released earlier this year.
However, Unity aims to roll out generative AI capabilities “over the next few months”.
Those shown in the demo video include motion synthesis and texture synthesis based on text prompts, and 2D sprite generation based on text inputs and inpainting.
Unity Sentis: embed AI models in the Unity runtime
Unity Sentis is aimed at developers, and makes it possible to embed an AI model into the Unity runtime itself, “enhancing gameplay and other functionality directly on end-user platforms”.
The teaser video, embedded above, suggests that it could be used to generate Ai-driven characters.
Price, release date and system requirements
Unity Muse and Unity Sentis are currently in invite-only beta. You can apply to join the beta program here.
Unity hasn’t announced a release date or system requirements for either service, or whether they will be available free to Unity users or via paid subscriptions.