After 12 years on the market, more than 238 million copies sold, numerous updates, mods and spin-off games, Minecraft remains one of the most popular titles for kids and anyone who is young at heart. Players can build houses, secret lairs or even in-game devices such as a computer that plays Minecraft (opens in new tab) (totally meta). But up until now, if you wanted to create anything in the game, you had to spend minutes or hours gathering the materials and putting them together. What if you could just ask a bot to do it for you?
According to Semafor Technology Editor Reed Albergotti, Microsoft has internally demoed a Minecraft feature (opens in new tab) which would allow players to use natural language commands to direct the characters or create things. Albergotti writes that, though the demo used AI, it was not powered by OpenAI's Prometheus Model, which underpins the Bing Chat service. However, he notes that "Microsoft has no immediate plans" to deploy this feature to a public build of Minecraft.
Albergotti cites "people familiar with the matter" as his sources for this information so it clearly falls under the category of a rumor or leak. However, it make sense that Microsoft would be testing AI technology in gaming and there's no better title to try it with than Minecraft, because it's a sandbox game where people constantly want to create things.
Microsoft would not be the first to use AI with Minecraft. In a research paper (opens in new tab) from June of 2022, OpenAI, the company behind ChatGPT and Bing Chat, showed how it had train a "behavioral cloning model" to perform actions in the game by showing it 70,000 hours of videos. The model was able to perform complex tasks such as building a crafting table, crafting a stone pickaxe or constructing a wood shelter.
We've also seen a number of individuals attempt to use ChatGPT with Minecraft. However, the fact that AI is not built into the game and its prompts (and that ChatGPT just outputs text) limits its functionality. For example, a YouTuber named SmallishBeans used ChatGPT (opens in new tab) to get advice about how to build an attractive house in the game, but he still had to do the actual work.
I asked my ten year old son, who plays Minecraft obsessively, what he would do with AI in the game and he said he would use it to build complex structures that otherwise take hours of manual labor to design. Having a bot take this work away raises interesting questions about the purpose of the game: is it to create with your own hands (or clicks) or just to see your ideas come to life? And if you ask the AI to create something for you and it makes lots of subtle design choices, whose design is it?
Some players would be excited by the possibility of avoiding the rote work of Minecraft: digging, chopping and constructing commonplace objects such as swords. Microsoft could also solve these "problems" by changing the mechanics of the game to automate these processes, but at what point is it no longer a rewarding play experience? If I train an AI to beat every level of Pacman for me, do I still feel a sense of accomplishment from playing the game?
We may be about to find out what affect natural language processing and advanced machine learning have not only on Minecraft but a slew of other games. Microsoft is well-positioned thanks to its ownership of key gaming IP, ranging from Flight Simulator to Halo.
Bots can obviously spoil or enhance a game, depending on who is using them and how. It just depends on the game.
I'd argue that there's a lot more downside to bots than upside, if the game is well-designed to begin with. IMO, it's not good design for games to require a lot of grind to level up or acquire resources. That's just game designers being lazy, or perhaps creating incentives for in-game purchases.
As a developer, I can see how having bots that can play your game could be useful for QA purposes, but only if the bots were quick & easy to train. Even then, they could never fully replace play testers.
On that last point (i.e. being "quick & easy to train"), we sort of come full-circle to my first point about it being rewarding to design an algorithm to win a game. Except, in this case, what strikes me as interesting is how they used a sort of 2-stage training strategy. In the graphic from the article, it shows how they trained one model to label a much larger dataset for training another algorithm. The basic concept isn't super novel, but I don't know enough to comment on which algorithms they used or how novel their application is, in this context.