Nvidia has released a festive video featuring a cute bobble head avatar of CEO Jensen Huang singing Jingle Bells. The traditional Christmas song has been jazzed-up 50s crooner style with some cool finger snapping action from the leather jacket draped ‘Toy Jensen. This fun video showcases cool technologies and spreads a little holiday cheer.
The video release isn’t just a way to say happy holidays to Nvidia fans, it also demonstrates (opens in new tab) the abilities of the Nvidia Omniverse Avatar Cloud Engine, or ACE, platform. The performance took “just hours,” to render and demonstrates realistic facial expressions and body movements alongside voice conversion technology which mapped Jensen’s tone to the performance of a pro singer – this isn’t an actual recording of Jensen’s singing voice.
Breaking down the video elements, the Nvidia development team used a recently developed model to map Jensen Huang’s voice tone onto a professional vocalist’s Jingle Bells singing sample. The musical notes from the professional singing were applied to the digital voice of Toy Jensen to make the avatar sing the correct notes and rhythm. It is notable that the musical arrangement for this performance wasn’t AI-generated, but was put together by Jochem van der Saag, a composer and producer who has worked with Michael Bublé and David Foster.
Next, Omniverse Audio2Face (opens in new tab) and Audio2Gesture (opens in new tab) technologies were leveraged to generate realistic facial expressions and gestures to match the Toy Jensen avatar’s animated musical performance. For a fuller X-Factor talent show style performance, we think Nvidia will have to develop some new Audio2Walk-down-steps technology.
Putting the above video into some perspective, Jensen Huang hit the headlines in November when he was visiting Taipei and video-bombed a singing duo. During a Taiwan Esports carnival event he managed to squeeze between Lah Yileh Lee and Xinting Lee during a performance of Lady Gaga’s “Always Remember Us This Way.” The performance went viral, but Huang insists he is too shy to record a real-person rendition of Jingle Bells, so tasked his engineers with readying the Toy Jensen performance, embedded top.
Nvidia says that the Omniverse ACE platform will be available soon for early access partners. It doesn’t say whether the release will include a Toy Jensen to play with.
