Nvidia Recreates Pac-Man Using AI

Pacman generated via GameGAN
(Image credit: Nvidia)

It's apparently the 40th anniversary of Pac-Man, and to celebrate Nvidia AI researchers cooked up a new tool called GameGAN and applied it to Pac-Man. The result: Without writing an engine or any code, the tool recreated Pac-Man and all of its rules with just a few days of training. But let's rewind and explain why this is cool and interesting.

GAN stands for generative adversarial network, basically two machine learning algorithms where one is trained to generate data that fits a model and the other is trained to determine whether the data is 'real' or not. GANs have been used in a variety of ways and have become increasingly popular in the machine learning world, and while Pac-Man is a relatively simplistic game by today's standards, the approach Nvidia used is applicable to more complex gaming environments and other real world scenarios. 

Nvidia trained the Pac-Man GameGAN using data from 50,000 games of Pac-Man, which were generated by a separate AI trained to play Pac-Man. The network received the input data (i.e., the directional keys that a player would press), along with the resulting video feed (images). Using just those pieces of data, GameGAN was able to learn and recreate the rules of Pac-Man and can now simulate the game—all without any direct coding of the game.

That means GameGAN knows that the yellow Pac-Man blob and the ghost blobs can't move through walls—not that it thinks of anything as a 'wall.' It understands that the ghosts move toward Pac-Man normally, but when Pac-Man eats a power pellet, they all turn purple and flee. That's because Pac-Man eats not just the dots, but also purple ghosts—and consumed ghosts turn into eyeballs that return to the center of the screen and then come back out.

What the GameGAN actually generates is a series of images that conform to its training data, which ends up being Pac-Man. It's pretty neat from a technical perspective, though things don't always go entirely as planned.

For example, the AI network that generated the 50,000 Pac-Man games for training is actually really good at Pac-Man, so it rarely died. That caused GameGAN to not fully comprehend that a normal ghost can catch Pac-Man and kill it. At one point, the network would 'cheat' and turn a ghost purple when it reached Pac-Man, or allow the ghost to pass through Pacman with no ill effect, or other anomalous behavio. Additional training is helping to eliminate this.

The GameGAN version of Pac-Man also targets a low output resolution of only 128 x 128 pixels right now. That's an even lower resolution than the original arcade game (224 x 288). 

I asked Nvidia if it could just use something like DLSS to upscale that to a better result. The answer was that many things are possible, but the researchers focused on getting the core training working first. For now, because properly rendering high-resolution images is far more difficult, we get a very pixelated Pac-Man game. That will likely change with future updates.

The fully playable GameGAN version of Pac-Man will eventually be available on Nvidia's AI Playground later this year. Other variations of Pac-Man are also possible with more training, which is likely part of what Nvidia is working on now before the public release. 

How long until GameGAN can recreate Crysis? Don't get your hopes up just yet, but at some point it could start creating new games and types of games. Cue the obligatory Skynet references.

Jarred Walton

Jarred Walton is a senior editor at Tom's Hardware focusing on everything GPU. He has been working as a tech journalist since 2004, writing for AnandTech, Maximum PC, and PC Gamer. From the first S3 Virge '3D decelerators' to today's GPUs, Jarred keeps up with all the latest graphics trends and is the one to ask about game performance.

  • AnimeMania
    I wonder what the file size difference is between a Human Generated Pac-Man game and an AI Generated Pac-Man game?
  • bit_user
    AnimeMania said:
    I wonder what the file size difference is between a Human Generated Pac-Man game and an AI Generated Pac-Man game?
    My guess is somewhere in the range of thousands to millions of time the size.

    Deep neural networks can be very large, while the original Pac Man game fit in only 16 KB of ROM for all the maps, sounds, characters, and game logic.

    Moreover, Pac Man ran on a 3 MHz 16-bit CPU, while I'm sure Nvidia's version requires a GPU millions of times faster.

  • DotNetMaster777
    Pac Man is nice game but generated game is with too low resolution !!
  • bit_user
    DotNetMaster777 said:
    Pac Man is nice game but generated game is with too low resolution !!
    Their point isn't that it's good, but rather that it's possible.

    You could think of it a bit like people who overclock CPUs using LN2 or who build dragsters that reach over 300 MPH in a quarter mile - they don't do it because it's a viable way to operate a computer or build a street car, but just to see how far they can push the technology.

    Nvidia is simply showing how deep learning can tackle ever more problems than it was originally conceived to be suited for. It's a little bit like a solution that's searching for problems, but worthwhile if they pioneer new techniques in the process. Hopefully, others will eventually employ these same techniques to do something a lot more useful.