Tobii Spotlight Technology Uses Eye Tracking to Lighten the Load for VR Headsets

Update 8/5/2019: Added details about GPU compatibility.

Original article 7/30/2019: 

Tobii, famous for its eye tracking technology used in security features like Windows Hello, is using its powers to lighten the workload of VR headsets and make the overall experience smoother. Tobii Spotlight Technology, announced Tuesday, uses eye tracking for dynamic foveated rendering (DFR), so VR headsets can focus on images in the center of the user’s focus, rather than what’s in their peripheral vision.

Foveated rendering is inspired by fovea, a small part of your retina that sees clearly while your peripheral vision is more blurred. Tobii Spotlight Technology uses DFR for better computation efficiency and to enable accurate, low latency tracking of what your eyes are looking at in real-time.

The technology may even be able to help people who get nauseous while in VR.

“Part of the nausea effect in VR can be due to an insufficient graphics refresh rate,” a Tobii spokesperson told Tom’s Hardware. “With DFR, it is possible to create graphics with higher and smoother frame rates that match display refresh rate and, thus, mitigate that part of the problem. Additionally, Tobii eye tracking is designed to help make VR devices better generally, including helping to better align the device for each user’s unique eye position (and thus provide a better visual experience).”

Tobii Spotlight Technology works with any graphics cards that support foveated rendering techniques. That includes the VRS (variable rate shading) tech used by cards based on Nvidia's Turing architecture. No AMD graphics cards currently support VRS, but a patent discovered in February suggests that some may do so eventually.

"We currently have not evaluated on AMD GPUs, but technically with an existing foveated rendering technique it should work," a Tobii spokesperson said. 

Tobii’s Benchmarks

Tobii Spotlight Technology is already available in HTC Vive Pro Eye, an enterprise-focused VR headset with built-in eye tracking. Tobii is bullish on its impact on VR headsets, based on benchmarks it shared using a Vive Pro Eye connected to a PC running an RTX 2070 graphics card and playing the game ShowdownVR with DFR enabled by Nvidia VRS and Tobii Spotlight Technology. Of course, we’ll have to take these results with a grain of salt, since it’s a vendor benchmarking its own technology.

According to Tobii’s testing, the average GPU rendering load decreased by 57% with an average shading rate of 16%.

With a lower GPU load, VR headsets, even high resolution ones, should have more headroom to maintain desirable frame rates.

Tobii also claims that this advanced eye tracking technique will be helpful as next-generation VR HMDs introduce even higher display resolutions. In fact, the greater the resolution, the more impact DFR has, according to Tobii’s benchmarks.

VR headsets using Tobii Spotlight Technology will also be able to use more advanced shading techniques without burdening the GPU load. This will purportedly allow developers to implement better lighting, colors, textures and fragment shaders.

What’s Next?

A company spokesperson told Tom’s Hardware that Tobii Spotlight Technology is “intended to support a variety of headsets, including both tethered and standalone headsets”.

The company believes that the next generation of foveating will enable things like foveated streaming (such as live content in low-latency 5G networks) and foveated transport, which uses data compression to optimize graphics transfer based on where the user is looking.

Photo Credits: Tobii