Yesterday, Microsoft Xbox debuted Muse, "a generative AI model designed for gameplay ideation," alongside an open-access Nature.com article and corresponding blog post with a YouTube video. Not sure what "gameplay ideation" means? While Microsoft defines it as generating "game visuals, controller actions, or both," its actual functional purposes are pretty limited and certainly don't do anything like skipping an actual game development pipeline.
That said, some of the data is still interesting. The training was done at scale on H100 GPUs. It required about 1 million training updates to extend just one second of real gameplay into an additional 9 seconds of responsive, engine-accurate simulated gameplay. Training data was primarily extracted from existing multiplayer gameplay sessions, though.
Instead of just running the game on a single PC, Microsoft had to train this model on a cluster of 100 Nvidia H100 GPUs, which is an order of magnitude more expensive and power-consuming while still only producing an output resolution of 300x180 pixels for about nine more seconds of extrapolated gameplay.
The most interesting thing the team demonstrated with Muse was duplicating existing props and enemies within the environment and having Muse replicate their functionality. All this hardware cost, electricity toll, and AI training instead of, you know, using development tools to spawn enemies or props instead?
While it's interesting how Muse eventually maintained proper object permanence and duplicated the original game's behavior, its final use cases seem downright wasteful compared to the already effective, traditional video game development pipeline.
While future developments of Muse may be capable of pulling off more interesting feats, it ultimately ends up on a long list of other projects trying to simulate gameplay entirely within AI. While it's nice to see some degree of engine accuracy and object permanence still present here, it's such a sub-optimal way to develop, test, or play a video game that I genuinely don't understand why anyone would want to use this, even after hours of poring over the relevant material.
