Nvidia's AI-Powered Eye Contact Feature is Terrifying and Hilarious

Testing Nvidia Broadcast's new Eye Contact feature
(Image credit: Tom's Hardware)

Last week, Nvidia launched a new version of Nvidia Broadcast (opens in new tab) — the deep learning and AI-powered software that can do noise suppression, background removal/replacement, camera framing, and now... Eye Contact. That last one is currently in beta, and... should probably stay in beta.

AI and deep learning has been in the news a lot lately, for good reason. Stuff like Dall-E, Midjourney, and Stable Diffusion are creating art from text, often with rather striking results. Of course, at other times you end up with mangled mutant creatures with two and a half heads and too many limbs. On the text side, ChatGPT is churning out legible writing that many fear means the death knell for English essays and journalism (and no, it did not write this news post).

The idea behind Eye Contact is simple enough: When you're on a webcast or meeting, often you look away from the camera. In fact, there's a real chance you're always looking away from the camera — because it's sitting at the top of the screen and the things you want to look at are on the screen. But what if there was a way to look like you're looking at your camera without looking at your camera?

What if you could train an AI model on faces and teach it to correct image where someone isn't looking straight into the lens? Get millions of images that are appropriately tagged, feed it into the network, and out pops an amazing tool, right?

Implementing it is not quite as simple; Nvidia has been talking about its Eye Contact feature for well over a year, and it's only now going into public (beta) release. Differences between myriad faces around the world makes it a tough problem to "solve," and even now the results are... imperfect (and that's putting it nicely). 

I went ahead and tested it anyway, on a system with an RTX 3090 Ti:

One of the things I noticed in testing is that often the live video feed would oscillate between me looking at the camera and me looking elsewhere, even though my focus stayed in the same spot. I guess this could be intentional, because having someone staring directly into the camera throughout an entire video chat would be a little creepy — but if it is, some adjustments to timing need to be made. 

What's more difficult to say is whether this sort of effect is even beneficial in the first place. If you want to look like you're looking at the camera, you should probably learn to look... at the camera. Solving human error through AI might just end up encouraging bad habits — what happens if you end up on a video feed that doesn't correct eye contact?

Regardless, Nvidia Broadcast with Eye Contact is now available for RTX owners to test. I tested it with an RTX 3090 Ti, but Nvidia lists the RTX 2060 as the entry point (and this should include mobile RTX 3050 GPUs, as far as I know). Long-term, I suspect at some point Nvidia will end up with some AI models that are more complex and require faster hardware than an RTX 2060 — just like how DLSS 3's Frame Generation feature requires an RTX 40-series graphics card — but for now any RTX GPU made in the past four years can power this feature. 

Do you like the effect, hate it, find it creepy, or something else? Let us know in the comments, along with other effects you'd rather see. I'm personally looking forward to the time when we can all have virtual cartoon avatars like Toy Jensen talking in place of real people, perhaps reading articles that were written by AI, with the videos and articles both being consumed by AI.

It's bots all the way down from there!

Jarred Walton is a senior editor at Tom's Hardware focusing on everything GPU. He has been working as a tech journalist since 2004, writing for AnandTech, Maximum PC, and PC Gamer. From the first S3 Virge '3D decelerators' to today's GPUs, Jarred keeps up with all the latest graphics trends and is the one to ask about game performance.

  • Jagar123
    I tested out the eye contact beta and it made my face look different. I looked like a different person with different eyes. This is in addition to the weird eye wiggle thing you were seeing. I am not very impressed with it as of yet.

    However, I do use the noise suppression within Nvidia Broadcast. Other than the app crashing every so often it has been fairly reliable for me.
    Reply
  • Elusive Ruse
    Hmmm... either your profile photo is too old or the eye contact feature changed your features a bit cause I didn't recognise you at first.
    Reply
  • JarredWaltonGPU
    Elusive Ruse said:
    Hmmm... either your profile photo is too old or the eye contact feature changed your features a bit cause I didn't recognise you at first.
    My YouTube profile photo is probably from 2015 or so, maybe even before that. LOL
    Reply
  • AgentBirdnest
    I just watched a few minutes of your video, Jarred. Skipped around a bit. Honestly, I didn't think it looked bad, for the most part. The only thing that creeped me out was that it looked like you had no irises half the time. The AI didn't seem to pick up your eye color, and it looked like you just had huge pupils with no color. That was in fullscreen though.
    Watching the small non-fullscreen video, the little quirks aren't so noticeable. Usually when I chat with someone, I make their video about 5 inches; not much bigger than the embedded video. I doubt I'd notice if someone had it on.
    However, I can imagine that if I were watching MYSELF on camera, it would probably be pretty weird. 😆 My webcam is borked and I haven't replaced it just yet, so can't try it right now. I look forward to it, though!

    At the same time, I don't think it would be necessary for my use case... whether chatting or recording an LP, I don't think anyone cares if I look around for a few seconds.
    It is certainly... interesting. Yeah. I'll just say "interesting", and that's all. 😀 I laughed at the part where you said you were looking straight forward, but the AI eyes (A-eyes?) kept shifting left and right. Hahahaha...
    Reply
  • Elusive Ruse
    JarredWaltonGPU said:
    My YouTube profile photo is probably from 2015 or so, maybe even before that. LOL
    Haha! I meant your photo from Tom's which accoompanies your brief bio at the bottom of each article but I assume that's from around the same time as your YT's :LOL:
    Reply
  • jp7189
    When you're presenting (like on YT), the making and breaking of eye contact is part of an engaging style. This is not for that in my opinion. However, there are people that are invited to 20+ person meetings all day long, and prefer to multitask during those meetings. This should be pretty good for those.
    Reply
  • RodroX
    Interesting, I still think the old glasses with fake eye stickers would do a better work lol
    Reply
  • Sarah Jacobsson Purewal
    AgentBirdnest said:
    Honestly, I didn't think it looked bad, for the most part.

    I think that's the problem, though, right? It doesn't look too bad...for the most part...but there's just enough there that it stays firmly in uncanny valley territory for me 🥴

    But, as Jarred pointed out yesterday, we've never met in real life! So, for all I know, Jarred is a robot.
    Reply
  • dankeldsen
    Like most tech denizens, I'm very interested in seeing where we can apply AI/ML to take away significant pain.

    To me, this is an interesting novelty, and as many people have noticed, it's creepy/hilarious a lot of the time.

    While this app (and those that are coming, or preceeded it







    >Zoom/comments/rrq7a1View: https://www.reddit.com/r/Zoom/comments/rrq7a1/any_webcam_experiences_with_center_cam_or_plexicam/

    We're now at over 20,000 sold, and are continuing to scale up fulfillment to meet the demand.

    No solution is right for everyone, but if you want a teleprompter-like approach, without the teleprompter pain, just take your favorite camera, put it in the middle of your screen (more or less), position your windows around the camera, and you're good to go!
    Reply
  • bit_user
    dankeldsen said:
    Or a teleprompter alternative like PlexiCam (I'm a co-founder), is another option.
    Did you send in a review sample? Maybe they (or Tom's Guide) will write a review of it.
    Reply