Last week at GDC, Oculus introduced a new rendering technique called Fixed Foveated Rendering (FFR), which reduces the GPU load in VR scenes. FFR is somewhat of a game-changer for mobile VR headsets, especially standalone units like the Oculus Go.
First, A Primer
Before you can truly understand the benefit of Fixed Foveated Rendering, you must first have a core understanding of Foveated Rendering, which is one of the short-term holy grails of VR advancement. Foveated Rendering is a rendering technique that lowers GPU overhead while maintaining a high level of visual fidelity by reducing the image quality outside your direct line of sight.
The concept of foveated rendering derives from biology—specifically, it leverages the natural operation of the Fovea in the human eye, which is a small portion of your retina with the clearest vision. Your peripheral vision is never as clear as what you see directly in front of you, and Foveated Rendering uses that concept to get the most out of your graphics hardware.
Ultimately, foveated rendering should enable ultra-high-resolution visuals in VR headsets without dramatically increasing the necessary rendering power. However, traditional thoughts on Foveated Rendering techniques suggest that eye-tracking technology is a fundamental key to enable Foveated Rendering. Eye-tracking cameras, such as the ones developed by Tobii, SMI, and Fove, can pinpoint where you’re gazing, which enables the application to dynamically adjust the visual fidelity of the area you’re focused on.
Pfft, Who Needs Eye-Tracking?
We’ve experienced Foveated Rendering demos from Fove and Tobii, and we were thoroughly impressed with the technology each time. Eye-tracking technology has come a long way in recent years, but it’s not yet available in consumer-ready VR headsets. Oculus isn’t waiting around for eye-tracking systems to catch up with the consumer VR market, though. The company is ready to capture the benefits of foveation, even if eye-tracking isn’t in the budget.
Last week at GDC, Oculus introduced Fixed Foveated Rendering (FFR), which features hard-coded zones with varying visual fidelity based on the likelihood of someone glancing directly in that area while playing a game or using an app. The central zone, where people most often look, gets rendered at maximum fidelity, but FFR reduces the clarity of the outer edges to reduce the GPU fill rate and conserve rendering resources.
Divide & Conquer
Qualcomm’s Snapdragon 821, which powers the Oculus Go headset, is a tiled renderer, which means it breaks its workload into rectangles and renders the pieces in parallel. FFR technology enables Oculus to dictate the resolution of each tile independently to ensure that the GPU's resources get directed where they can provide the biggest benefit.
Oculus’ FFR technology sub-divides the image destined for the headset’s display into 19 sections of varying resolution. The central area, which is where your eyes would normally be focused, receives no resolution restriction. In fact, it’s possible for the perceived resolution to increase in the central zone when FFR is implemented in older content from the Gear VR platform.
The other 18 sections surround the full-resolution central zone, with higher resolution tiles near the bottom of the display compared to the top. The diagram above demonstrates the different resolution targets of the surrounding rendering zones and their distribution. Red tiles equal 1/2 the resolution of the central zone, green tiles are 1/4 resolution, blue tiles are 1/8 resolution, and the magenta spots in the upper corners are just 1/16 resolution.
Not A Fix-All Solution
As much of a breakthrough FFR is, it’s not the be-all-end-all rendering solution. FFR works great for low contrast textures, such as background images and large objects, but it doesn’t work well for high-contrast items, such as text. To get around the blurry text issue that FFR presents, Oculus created a tool for developers that enables them to fine-tune the FFR level for the best performance and visual quality at up to a frame by frame rate.
Big Performance Gains
The lower resolution zones help reduce GPU fill requirements, but Oculus also uses it to solve a massive rendering inefficiency that plagues VR platforms: VR headsets feature lenses that warp the image to increase the perceived field of view. To make the image look normal, the runtime software pre-warps the image so that the lens corrects the visual. Unfortunately, the warping process incurs a heavier than necessary rendering burden on the outer edges of the screen, which wastes valuable GPU resources on areas of the screen that are seldom seen.
Oculus uses FFR technology in the Oculus Go to push low resolution tiles to the outskirts of the post-warped image, and the effect can have a significant impact on rendering performance. Oculus said that it observed as much as a 25% performance increase in pixel-intensive titles when FFR is enabled. And the benefits transcend game engines--Oculus said that FFR works with Unity, Unreal Engine 4, and “native applications.”
Oculus also noted that almost all mobile SoC platforms are tiled renderers, which suggests that FFR technology, or something similar, could materialize on other mobile and standalone VR platforms. This technology could be instrumental in bringing a new era of ultra-high resolution standalone VR devices to light in the future.
