Developers often include dimwitted bots in training modes, single-player recreations of multiplayer game modes, and other parts of a game. These automatons rarely exhibit good decision making; most of us have probably watched bots run into walls or spin in circles. DeepMind's artificial intelligence is a bit more sophisticated, and on May 30, the company revealed that its bots can beat human players in Quake III's Capture the Flag mode.
DeepMind is no stranger to developing machines capable of serving humans a heaping helping of humble pie. The most well-known example is AlphaGo, which beat the world's best Go player so handedly that he said competing against the AI was like playing in his backyard while AlphaGo explored the universe. The company has also worked to teach AI agents to play StarCraft II. (Another organization, OpenAI, taught its agents Dota 2.)
Quake III is wildly different from Go. It's a three-dimensional game played in a first-person perspective, to start, and its Capture the Flag game mode is team-based rather than a one-on-one competition. DeepMind also chose a variant of the mode that relies on procedurally generated maps, which meant the agents couldn't simply learn the best solution for a pre-defined space, and instead had to learn and master the principles of Capture the Flag itself.
To give you an idea of how well the agents learned how to play Quake III: DeepMind dubbed these agents For The Win (FTW) and they definitely earned that moniker. Despite knowing nothing about Capture the Flag to start, they learned the rules of the game as well as the core principles for success pretty quickly. Eventually they overtook 40 human players in a tournament where agents and humans played with and against each other. For science.
DeepMind conceded that some of FTW's success could be attributed to the agent's literally inhuman reaction times. (Again: most of us have probably wondered how the heck a bot managed to kick our butts no matter how hard we tried to win.) So it slowed them down and, sure, enough, the agents still beat their human counterparts. To rub salt in the wound, the company said most players also rated FTW as more collaborative than humans.
That means it's probably time to stop calling bad players bots. FTW showed that AI could learn the rules of a game, grok the fundamentals, and then get so good at the game that it beat humans even when it was handicapped with reaction times comparable to our own. Since these bots are actually better teammates than most humans, too, it seems like calling someone a bot could actually be a compliment once these agents go mainstream.
What It Means for AI
None of which is to say that FTW's success at Capture the Flag--and later other Quake III game modes--was just fun and games. DeepMind said that the work on this project "highlights the potential of multi-agent training to advance the development of artificial intelligence: exploiting the natural curriculum provided by multi-agent training, and forcing the development of robust agents that can even team up with humans."
The company also noted that FTW learned how to play Capture the Flag in (perhaps surprisingly) human-like ways:
"Through unsupervised learning we established the prototypical behaviours of agents and humans to discover that agents in fact learn human-like behaviours, such as following teammates and camping in the opponent’s base. [...] These behaviours emerge in the course of training, through reinforcement learning and population-level evolution, with behaviours–such as teammate following–falling out of favour as agents learn to cooperate in a more complementary manner."
More information about this project is available via DeepMind's website, a paper that was published in Science, and a YouTube video on the subject from 2018.
