Microsoft Xbox showcases an incredibly impractical AI model for 'gameplay ideation'

Microsoft's Muse AI being used to simulate inserting extra props and enemies into an existing scene from the game Bleeding Edge.
Microsoft's Muse AI being used to simulate inserting extra props and enemies into an existing scene from the game Bleeding Edge— these soon become fully interactable as expected based on existing training data. (Image credit: Microsoft, Ninja Theory)

Yesterday, Microsoft Xbox debuted Muse, "a generative AI model designed for gameplay ideation," alongside an open-access Nature.com article and corresponding blog post with a YouTube video. Not sure what "gameplay ideation" means? While Microsoft defines it as generating "game visuals, controller actions, or both," its actual functional purposes are pretty limited and certainly don't do anything like skipping an actual game development pipeline.

That said, some of the data is still interesting. The training was done at scale on H100 GPUs. It required about 1 million training updates to extend just one second of real gameplay into an additional 9 seconds of responsive, engine-accurate simulated gameplay. Training data was primarily extracted from existing multiplayer gameplay sessions, though.

Instead of just running the game on a single PC, Microsoft had to train this model on a cluster of 100 Nvidia H100 GPUs, which is an order of magnitude more expensive and power-consuming while still only producing an output resolution of 300x180 pixels for about nine more seconds of extrapolated gameplay.

Introducing Muse: Our first generative AI model designed for gameplay ideation - YouTube Introducing Muse: Our first generative AI model designed for gameplay ideation - YouTube
Watch On

The most interesting thing the team demonstrated with Muse was duplicating existing props and enemies within the environment and having Muse replicate their functionality. All this hardware cost, electricity toll, and AI training instead of, you know, using development tools to spawn enemies or props instead?

While it's interesting how Muse eventually maintained proper object permanence and duplicated the original game's behavior, its final use cases seem downright wasteful compared to the already effective, traditional video game development pipeline.

While future developments of Muse may be capable of pulling off more interesting feats, it ultimately ends up on a long list of other projects trying to simulate gameplay entirely within AI. While it's nice to see some degree of engine accuracy and object permanence still present here, it's such a sub-optimal way to develop, test, or play a video game that I genuinely don't understand why anyone would want to use this, even after hours of poring over the relevant material.

Christopher Harper
Contributing Writer

Christopher Harper has been a successful freelance tech writer specializing in PC hardware and gaming since 2015, and ghostwrote for various B2B clients in High School before that. Outside of work, Christopher is best known to friends and rivals as an active competitive player in various eSports (particularly fighting games and arena shooters) and a purveyor of music ranging from Jimi Hendrix to Killer Mike to the Sonic Adventure 2 soundtrack.