Wednesday, March 22nd, 2023 Posted by Jim Thacker

Unity ‘unveils’ Unity AI


Posted on 22 March 2023. Unity has now announced AI services Unity Muse and Unity Sentis.

Unity has posted a teaser video for Unity AI, a mysterious initiative it describes as “building an open and unique AI ecosystem that will put AI-powered game-development tools in the hands of millions of creators”.

The video, which promotes a new beta program, was released during GDC 2023.

Promoting a new beta program, probably for developers of third-party AI tools
Generative AI art tools have been the flavour of the week, with Adobe announcing Firefly, its suite of online AI art tools, including a text-to-image generator, and Nvidia making several more announcements, including an upcoming text-to-3D model generator, developed in partnership with stock asset library Shutterstock.

Unity’s video is considerably more enigmatic, showing a user typing in plain text prompts like, ‘Give me a large scale terrain with a moody sky’ and ‘Add a dozen NPCs… make them flying alien mushrooms’.

Sadly, the video doesn’t actually show the large scale terrain or the flying alien mushrooms, so we have no idea in what form they would be generated inside the game engine – if, indeed, they are generated at all.

However, it may refer to a program for third-party developers, rather than something Unity is developing itself: according to Reuters, Unity “aims to open a marketplace for generative AI software”.


A demo video from Unity’s latest blog post on its vision for AI in the Unity Editor and runtime. It isn’t captioned, but seems to show an AI-assisted character posing system.


Updated 18 May 2023: Unity has published a blog post on its work on AI.

It doesn’t give any details on which tools will become available through the Unity AI beta program, but it does provide more context on the company’s approach to AI.

Among other things, it says that “Unity creators will be able to use natural-language generative tools … to create code, animations, physical effects, or other real-time content”.

The blog post also notes that Unity “is building the technology in the Unity Editor to better define what AI draws from” including “using appropriate and licensable datasets”.

Perhaps most interestingly, it revels that Unity has been working on integrating AI technology, codenamed ‘Barracuda’, in the Unity runtime for “more than five years”.

When rolled out publicly, the technology will make it possible for game developers to use machine learning techniques to create gameplay loops, as well as generating art assets.

Potential use cases listed in the blog post include “NPCs that come to life, diffusion content as a gameplay mechanism [and] boundaryless user-generated content”.

Register for the Unity AI beta program