A diverse group of researchers from Nvidia, Stanford, Caltech, and other institutions has introduced NitroGen. In a LinkedIn post on Friday, Jim Fan, Nvidia Director of AI & Distinguished Scientist, heralded NitroGen as “an open-source foundation model trained to play 1000+ games.” However, the implications are much wider, spilling from game worlds into the real world, with sizable benefits for simulations and robotics.
You could say, this research presents an attempt to distill a ‘GPT for actions.’ Thus, it is a kind of LLM breakthrough, applying this proven large-scale training tech beyond the fields of language and computer vision. Moreover, the pioneering building of “generally capable embodied agents that can operate in unknown environments has long been considered a holy grail of AI research,” asserts an introduction to the research paper.
Interestingly, NitroGen’s foundation is the GROOT N1.5 architecture, originally designed for robotics. And its application inside the world of gaming shows potential to circle back and bring great benefits to robots working in diverse or unpredictable environments, too.
NitroGen was adapted to play games packed with wildly different mechanics and physics – that’s the nature, and fun, of video games. The researchers used 40,000+ hours of public gameplay videos shared by streamers. Videos in which gamers overlaid their real-time gamepad interactions on the stream were particularly helpful.
In tests, NitroGen was successful in games as diverse as “RPG, platformer, battle royale, racing, 2D, 3D, you name it!” enthuses Fan. While results are promising, the Nvidia scientist says that this is just the start, with a big hill left to climb.
This first version of NitroGen is intentionally focused on fast motor control, or ‘gamer instinct,’ as Fan calls it. According to the shared research, the new LLM also has a “strong competence across diverse domains,” and the model works in procedurally generated worlds as well as unseen games with a “52% relative improvement in task success rates over models trained from scratch.”
All the research into NitroGen so far has been open sourced, and those interested in gaming, robotics, and LLMs are encouraged to tinker. Tweaks to the pretrained model weights, the entire action dataset, and code are all open to your flights of fancy and fiddly digits.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.
