How We Tested
Typically, we'd use our GPU test system for VR benchmarking. It consists of an Intel Core i7-5930K, 16GB of Crucial Ballistix Sport DDR4 memory, and a pair of Crucial M400 SSDs. But I wasn’t able to launch the VRTrek Test on that machine. The MSI X99S XPower AC motherboard we use does feature Realtek audio, but VRScore couldn't read a signal from the VRTrek latency testing tool. After a day of troubleshooting, I switched to an older PC with an Intel Core i5-4570K, 16GB of Crucial Balistix Sport DDR3, an Asus Z87-WS motherboard, and a 500GB Samsung 850 EVO SSD, all driven by a Corsair RM850 power supply.
Test System Specs
We ran VRScore with a Gigabyte GTX 1080 G1 Gaming graphics card and the PowerColor RX 480 Red Devil 8GB. The scope of this piece is to demonstrate how VRScore works, not to compare GPUs. That story is coming soon. And yes, it's going to be hot.
Instead of going wide on graphics cards, I chose to include as many HMDs as possible. The line-up of platforms available to me includes an Oculus Rift CV1, a Rift DK2, an OSVR HDK 1.4, and three different Vives (our Vive Pre, the retail Vive we received for launch coverage in April 2016, and a new Vive I purchased in December 2016, which includes the new single-cable data tether).
|Test HMDs||HMD Resolution||HMD Refresh Rate||VRScore Test Resolution|
|Oculus Rift DK2||1920x1080 (960x1080 per eye)||75Hz||2664x1586 (1332x1586 per eye)|
|Oculus Rift CV1||2160x1200 (1080x1200 per eye)||90Hz||2364x1464 (1182x1464 per eye)|
|HTC Vive Pre||2160x1200 (1080x1200 per eye)||90Hz||3024x1680 (1512x1680 per eye)|
|HTC Vive (launch day)||2160x1200 (1080x1200 per eye)||90Hz||3024x1680 (1512x1680 per eye)|
|HTC Vive (purchased Dec 2016)||2160x1200 (1080x1200 per eye)||90Hz||3024x1680 (1512x1680 per eye)|
|OSVR HDK 1.4||1920x1080 (960x1080 per eye)||60Hz||2880x1620 (1440x1620 per eye)|
We ran the System Test on each HMD, with each GPU, with and without an HMD hooked up. Basemark’s evaluation guide recommends running the VRTrek test at least five times and averaging the results. Its guidelines also suggest that you should discard any result with more than a 2% variance from the average.
VRScore works with each of the primary VR runtime environments, enabling compatibility with Oculus’ Rift, Valve/HTC’s Vive, and OSVR’s Hacker Development kits. Basemark warns that you shouldn’t have more than one VR runtime installed at once to avoid interference. For example, if you have an Oculus Rift with the Home software and SteamVR drivers installed, VRScore may launch the incorrect runtime.
The need to remove drivers corresponding to other HMDs is an issue I wish Basemark had addressed more thoroughly. After all, uninstalling the Oculus Home software is downright inconvenient since downloaded content is tied to the Oculus install folder. Deleting that eliminates however many tens of gigabytes of content you had, too.
Uninstalling the Oculus software caused other problems, too. For some reason, after removing Oculus Home from the test system, we were unable to reinstall it. The installation routine kept kicking back an error and prompting for a reboot. This issue isn’t directly related to Basemark’s utility, but it’s a documented bug without a simple solution. We eventually had to reformat and start anew.
After reinstalling Windows 10, I left Oculus Home installed while testing one of the Vive HMDs and successfully ran the benchmark. Just remember to remove SteamVR before launching VRScore on the Rift. OSVR's runtime wasn’t affected by Oculus Home either. But again, I had to remove SteamVR to perform the test correctly.