Valve Index VR: Gaming Benchmarks
A Wider Window Into the Virtual World
Valve’s Index headset features two LCDs with a per eye resolution of 1440 x 1600 (Valve has not disclosed the size of the panels). That's the same resolution as the HTC Vive Pro and Samsung’s Odyssey (Plus) headsets. However, you get a more full FOV with the Index .
Valve designed custom dual-element lenses for the Index. It said that these lenses provide “high geometric stability” and “minimal shape distortion,” which enables Valve to “maximize FOV without sacrificing edge-to-edge clarity.” In other words, the Index can make use of more of its display than older headsets.
Valve also installed the displays in the Index at a 5-degree outward cant to maximize the peripheral view. The company said that the difference in FOV between the Index and Vive could be as significant as 20 degrees for typical users.
I have a 6mm VR Cover installed on my Vive, which improves the FOV, so the difference wasn’t as pronounced as it could have been from my perspective. But the Index still offers a broader view of the virtual world than the Vive. Pimax headsets, like the Pimax 5K Plus and Pimax 8K, remain your best option for the broadest FOV.
RGB Subpixels Make a Huge Difference
The Index has the same resolution as the Vive Pro and Odyssey, but not the same display technology. Whereas the Vive Pro and the Odyssey feature AMOLED displays, which produce excellent contrast and deep blacks, the Index features ultra-low persistence LCDs. While it’s true that the LCD panels can’t produce true blacks because they require backlighting to function, LCD panels can take advantage of RGB subpixel arrays that improve image sharpness and eliminate screen door effect.
Valve claimed that the RGB LCDs provide “50% more subpixels than OLED” displays, which results in “three times better” fill-factor, and I’m inclined to agree. The image clarity in the Index headset is second only to HP’s 4K per eye Reverb headset. I would say it provides better image clarity than my Pimax 5K Plus
80 Hz, 90 Hz, 120 Hz, or 144 Hz? The Choice Is Your GPU’s!
The LCD panels in the Index also provide another distinct advantage over OLED panels: They’re available with higher refresh rates. Valve’s Index is the first PC-VR headset to offer panels that support 120 Hz and even 144 Hz operation, although the latter is an experimental feature.
The SteamVR software enables you to select 80 Hz, 90 Hz, 120 Hz or 144 Hz, which allows the device to scale with your computer. If your PC isn’t quite up to par for the higher refresh rates, you can dial the settings back until your next system upgrade. To be honest, you’ll likely need to keep the settings dialed back for a while. 90 Hz is already a pretty tall order for most GPUs in many games. You will need a top-tier GPU to run your Index at 120 Hz or higher, as our benchmark results illustrate.
Ready for the Future
Valve is so serious about making the Index a premium device that it even planned for future expansion and advancements. The front faceplate of the Index headset features a removable panel that covers a cavity that Valve calls the “Frunk.” Valve doesn’t have any specific plans for the front compartment but put a USB port there so that developers and hardware makers can experiment with it. Some have assumed the Frunk would fit a Leap Motion controller. It true you can slip one in the slot, but there’s not enough room for the USB cable, so it doesn’t quite fit.
Valve also installed a pair of front-facing, global-shutter RGB cameras on the Index. Currently, the cameras don’t have a specific use, but Valve said they were designed for computer vision and they're looking forward to computer vision experts to make use of them. At the very least, we hope to see the adoption of stereo passthrough for augmented reality experiences. Valve said it would have more to share about the cameras later this summer.
As usual, my VR headset test system consists of an Intel Core i7-8700K CPU overclocked to a clock speed of 4.8 GHz, 16GB of G.Skill SniperX DDR4-3600 RAM, a Crucial 500GB MX200 SSD and an Nvidia GeForce RTX 2080 Founders Edition graphics card.
For this evaluation, all tests were conducted with version 430.86 of Nvidia’s Game Ready GeForce driver installed.
Usually, I would include results for at least four games, but due to a series of mishaps, including a variety of computer-related setbacks, I had to cut the number of games to two. We'll update this review when we have additional benchmarks.
We use Nvidia’s FCAT VR tool to capture the real-time frame rate and frame time data from SteamVR. For each variable change, we capture three 60-second tests. I chose to focus my attention on the adjustable refresh rate rather than adjust the display resolution. I set SteamVR’s rendering scale to 100%, which resulted in 2016 x 2240 pixels per eye.
I focused on 90 Hz, 120 Hz and 144 Hz. I ignored the 80 Hz mode because the RTX 2080 had no trouble with 90 Hz, so reducing the frequency served no purpose. I also ran tests with motion smoothing enabled and disabled to see what effect it has on performance—both objectively and subjectively.
Serious Sam VR: The Last Hope
Serious Sam VR: The Last Hope is a fast-paced gallery shooter with intense waves of dozens and even hundreds of enemies on screen at once. It is the type of game that can take advantage of the fast refresh rates that the Index offers.
I allowed the game to autodetect the hardware configuration, which rated the CPU and memory as high, and the GPU as medium. The game automatically enabled 125% render scale, which I kept for this test. The rendered resolution ended up being 2253 x 2504.
At 90 Hz, the RTX 2080 had no trouble keeping up with the workload. The average unconstrained frame rate was more than double the required 90 frames per second (fps), which meant that even with motion smoothing enabled, the frame rate never dipped below 90. I didn’t perceive an immediate difference between motion smoothing off or on at 90 Hz.
With the display adjusted to 120 Hz, the effects of motion smoothing were much more pronounced. With motion smoothing enabled, the average delivered fps was significantly lower than the displays’ refresh rate at 102.39 fps because motion smoothing cuts the unconstrained 206.35 fps right down the middle. That resulted in over 1,100 missed frames out of 7,984, which is 13.8% loss.
With Valve’s smoothing technology disabled, the GPU cranked out an average of 118.74 fps, which works out to just over 4% loss.
More importantly, and ironically, the game felt much smoother with motion smoothing disabled, and I felt like I was much more accurate when I didn't continually see synthetic frames.
The pattern continued when I bumped the refresh rate up to the experimental 144 Hz mode. With this frequency, the GPU struggled to keep the frame buffer filled. With motion smoothing on, the GPU delivered 119.23 fps, and with motion smoothing off, it averaged 138.4 fps.
Again, while playing the game with motion smoothing enabled, I could tell that something was off with my aim. My hit rate dramatically improved once I switched it off.
Space Pirate Trainer
In the week leading up to the embargo lift, our Index headset started to malfunction. SteamVR kicked an error that indicated that it couldn’t detect the headset after rebooting the software to initiate a refresh rate change.
After some troubleshooting, I got the headset to work again by reseating the display cable at the headset end. Following that, SteamVR kept giving me false errors about the tether being disconnected at the headset end even when the display was working just fine. If I would leave the headset idle for a few minutes, I'd get an error about the headset not being detected. Sometimes reseating the cables would work, and sometimes it would require a complete power down of the PC to get it working.
Eventually, the headset stopped responding to my quick fixes, and I had to leave it sit. I believe it may have been overheating as the left side of the Index and the tether cable were warm to the touch. I tried it again the next day, and it turned on at the first try.
But currently, the headset shuts off while in use. The displays go gray, and SteamVR kicks an error about not being able to see the controllers.
I’m Probably an Outlier
I hope that these issues are unique to my unit, and I have reason to believe that they may be. I’m in North America but am running a European version of the Index. Because of that, I’m using an adapter to change the power plug to work with my outlets. I suspect this could be causing a voltage issue or there could be poor wiring in the adapter. Or, I could have pulled the short straw and received a rare dud. It’s hard to tell right now, and I'll update this article when I can determine what the problem is.
Still, I can say I won’t be using HTC’s wands anymore. Valve won me over with the Index controllers.
Valve didn’t initially plan to make its own VR headset, but developed the class-leading Lighthouse tracking technology, made licenses available for free and working with HTC, which licensed both generations of Valve’s tracking system. However, HTC appears to be moving away from Valve’s base station-based tracking solution in favor of inside-out tracking (no base stations required), as found on theVive Focus Plus and Cosmos headsets.
Rather than allow its research and development investment to go to waste, Valve chose to move forward without a partner. The company developed the Index headset in-house, and as a result, it was able to control every aspect of the Index’s design, which enabled it to implement several innovative design choices that will surely inspire headset design for years to come.
If there's a better VR headset for consumers, I haven't tried it yet. There is certainly room for improvement with the Index—most notably, I would like to see it go on a diet. If you have the money to burn, and you want the best home VR experience that money can buy, the Index headset combined with the controllers delivers a champion-level consumer VR experience.
Image Credits: Valve, Tom's Hardware
MORE: Virtual Reality Basics