Ubisoft, Nvidia, and Inworld AI partnership to produce "Neo NPC" game characters with AI-backed responses

Ubisoft's Neo NPC project in action, which leverages AI for player interaction.
Ubisoft's Neo NPC project in action, which leverages AI for player interaction. (Image credit: Ubisoft)

During this year's Game Developer's Conference (GDC 2024), Ubisoft has unveiled the product of a collaboration between itself, Inworld AI, and Nvidia's Audio2Face tech, demonstrating "Neo NPCs" capable of speaking directly to human players. This is made possible through the mighty, sometimes-terrifying, sometimes-quite-underwhelming power of generative AI, and is something we've been expecting to see since Inworld first popped up as an Xbox partner last year.

Interestingly, this actually isn't the first-ever example of interactable AI NPCs. An indie murder mystery game on Steam titled Vaudeville released on Early Access in June of 2023 and made significant waves for its litany of conversation-capable NPCs.

Another aspect focused on with "Neo NPCs" is that they can still have pre-written backstories and motivations. In fact, that's leveraged to help keep the character and their answers immersed in the setting, despite whatever the player might be saying that goes against those goals. Narrative Director Virginie Mosser said, "It's very different. But for the first time in my life, I can have a conversation with a character I've created. I've dreamed of that since I was a kid."

As genuinely cool as all of that sounds, we don't want to understate the potential negative impact this could have on actual games writing and voice acting moving forward. Compelling voice and writing talent have been responsible for some of the most memorable moments in video games for decades— to such an extent AAA devs have been doing facial capture with real-life prestige actors for quite some time, now.

Christopher Harper
Contributing Writer

Christopher Harper has been a successful freelance tech writer specializing in PC hardware and gaming since 2015, and ghostwrote for various B2B clients in High School before that. Outside of work, Christopher is best known to friends and rivals as an active competitive player in various eSports (particularly fighting games and arena shooters) and a purveyor of music ranging from Jimi Hendrix to Killer Mike to the Sonic Adventure 2 soundtrack.

  • hotaru251
    wanna bet they list games that sue this as "AAAA" even though it wont really add any benefit to player? (very few games have any reason to talk to an npc)
    Reply
  • Sleepy_Hollowed
    Unless models and ML processors are forever backwards and forward compatible, this is a very expensive tax write off.

    I wouldn’t touch this as a dev unless the ML can be ran on python as software emulation on-computer.
    Reply
  • Notton
    It's noobisoft, they are going to over promise and under deliver, like they always have.

    I am honestly surprised they haven't been under investigation for fraud and stock manipulation, etc.
    Reply
  • baboma
    VTi2l_hjVRsView: https://youtu.be/VTi2l_hjVRs
    Dialog for the 1st NPC (Bloom) is pretty on-point, the 2nd one not so much. My question would be how much storywriting work is required to get to this level of conversation for Bloom, keeping in mind that this is just one NPC out of potentially many.

    The text-to-speech can probably be done with on-device NPU, but the LLM response at this point would probably require cloud-based connection (read: Internet connection). This would likely entail upsell to a recurring payment model (read: subscription).

    Personally I would rather just to have the responses conveyed in text form, and skip the T2S along with facial expression & lip sync effects. It's less work, and believability/immersion wouldn't be any worse than that shown in the demo. Even if the demo's response lag & stuttering were fixed, it would still feel like talking to an animated mannequin.
    Reply
  • jeffy9987
    hotaru251 said:
    wanna bet they list games that sue this as "AAAA" even though it wont really add any benefit to player? (very few games have any reason to talk to an npc)
    no AAAA is for live service games AAAAA is for games POWERED BY AI or maby AIAAAA patent pending
    Reply
  • tamalero
    baboma said:
    VTi2l_hjVRsView: https://youtu.be/VTi2l_hjVRs
    Dialog for the 1st NPC (Bloom) is pretty on-point, the 2nd one not so much. My question would be how much storywriting work is required to get to this level of conversation for Bloom, keeping in mind that this is just one NPC out of potentially many.

    The text-to-speech can probably be done with on-device NPU, but the LLM response at this point would probably require cloud-based connection (read: Internet connection). This would likely entail upsell to a recurring payment model (read: subscription).

    Personally I would rather just to have the responses conveyed in text form, and skip the T2S along with facial expression & lip sync effects. It's less work, and believability/immersion wouldn't be any worse than that shown in the demo. Even if the demo's response lag & stuttering were fixed, it would still feel like talking to an animated mannequin.
    Not to mention has a game function goes.. generating multi branches for conversations would be incredible tiresome if you're just looking to complete a quest and get to the point.

    This whole dynamic convos would be somewhat fun for completely random non essential NPCS.
    Reply