Skip to main content

Basemark's VRScore Benchmark Suite: 6 VR HMDs Tested

Performance Tests

Make sure your HMD is connected to your PC, launch VRScore, and select the test you wish to perform. If you start the benchmark before plugging in an HMD, VRScore will launch in desktop mode. If the software detects a Rift or Vive, it will also find the corresponding driver on its own. However, you must launch the OSVR Server before VRScore to measure an OSVR HMD. And if you want to run the VRTrek Test, make sure you have the VRTrek latency tool plugged into the mic input jack on your motherboard first.

The Official System Test scene features high-resolution textures far beyond the resolution of modern HMDs. Again, the software allows the runtime to set its resolution, making it difficult to compare headsets to each other. Still, Basemark calculates the results of each test to generate a number used for comparing to other systems. 

The Oculus, HTC, and OSVR headsets operate at different resolution settings in VRScore. SteamVR facilitates the highest resolution; although the Vive natively runs at 2160x1200, Valve’s runtime allows settings as high as 3074x1680. The Rift CV1 offers the same visible resolution as the Vive, but Oculus' runtime sets VRScore's resolution to 2664x1586. Both resolutions exceed what you actually see on-screen, but remember VR is rendered to a larger target. What Valve sets as a target requires more horsepower in order to maintain smooth performance. Consequently, the Rift CV1 consistently returns higher scores than the Vive.

Image 1 of 8

Image 2 of 8

Image 3 of 8

Image 4 of 8

Image 5 of 8

Image 6 of 8

Image 7 of 8

Image 8 of 8

The Oculus Rift DK2 sets a higher resolution too, but it's lower than other HMDs. As a result, we see the DK2 return a better VRScore. Oculus’ second developer kit featured a single 1080p panel with a 75Hz refresh rate, which explains why it doesn't need the same class of hardware as the consumer headsets. The single display in the headset also causes each eye to update at a different pace, which affects score.

"Some of the older HMDs still utilise rolling display update (right-to-left, up-to-down) instead of just global one time flash for whole display," said Arto Ruotsalainen, Basemark's Co-Founder and CEO. "When using VRTrek with rolling displays the measurements can vary depending where the sensors are pointing. VRScore PC with VRTrek will give you Application-to-Photon latency measurement. This is the time from submit of image to HMD to the time when it actually appears as a photon.In general, there are two kinds of displays: global flash (all pixels at the same time) and rolling display (up-to-down, right-to-left, etc). All major brands already have global flash (HTC Vive, Rift) as rolling displays also affect to user experience (nausea, looks like wobbling effect)."

Our sample GPUs come from different performance classes. Gigabyte's GTX 1080 G1 Gaming has no trouble maintaining roughly 90 FPS with the Rift and Vive, but PowerColor's RX 480 struggled to keep up. Conversely, AMD's GPU didn’t have any trouble keeping up with the DK2's 75Hz display.

Image 1 of 6

Image 2 of 6

Image 3 of 6

Image 4 of 6

Image 5 of 6

Image 6 of 6

VRScore is compatible with OSVR, so we included our OSVR HDK 1.4 kit featuring a 60Hz, 1080p display.

After the DK2 kicked back a higher score than the Vive or Rift CV1, we were surprised to see the OSVR kit return a lower result. OSVR's runtime sets a resolution of 2880x1620 pixels, which is quite a bit higher than the DK2's chosen resolution. Our GeForce GTX 1080 does deliver ample performance for this test. Unfortunately, though, the OSVR headset refused to interface with AMD's Radeon RX 480. OSVR’s driver indicates support for AMD graphics cards, but the headset would not activate in Direct or Extended mode. AMD did release a new software package the day after we received the review copy of VRScore, so perhaps it introduced an incompatibility.

Image 1 of 6

Image 2 of 6

Image 3 of 6

Image 4 of 6

Image 5 of 6

Image 6 of 6

The VRTrek test includes part of the System Test's VR sequence to measure dropped and duplicated frames. It also features the aforementioned latency test to determine application-to-photon latency. The latency test sends pulses of white light to the headset’s display, and the VRTrek device records when the signal reaches the display. The process is similar to the latency test from Futuremark, which we tested more than one year ago.

Image 1 of 4

Image 2 of 4

Image 3 of 4

Image 4 of 4

The results page in VRScore doesn’t yield a lot of information, but Basemark provides a full report for each test that you can view online. These numbers feed into a bar graph that compares where your system stands against other configurations, and line graphs that indicate Per Frame Results, Time Taken By Frame Submit, and From Submit To Photon metrics. Basemark allows you to zoom in on the three line graphs to get a better perspective of the data.


MORE: Best Virtual Reality Headsets


MORE: The Sony Playstation VR Review


MORE: Oculus Rift Vs. HTC Vive Vs. PlayStation VR

  • Piki__
    Comparing GTX 1080 vs RX 480? HOw unfair can you be. Compare to GTX 1060!
    Reply
  • kcarbotte
    19234404 said:
    Comparing GTX 1080 vs RX 480? HOw unfair can you be. Compare to GTX 1060!

    This isn't a GPU comparison. The article's scope is to show what VRScore is all about.
    Reply
  • Piki__
    19234670 said:
    19234404 said:
    Comparing GTX 1080 vs RX 480? HOw unfair can you be. Compare to GTX 1060!

    This isn't a GPU comparison. The article's scope is to show what VRScore is all about.

    Reply
  • kcarbotte
    19234733 said:
    19234670 said:
    19234404 said:
    Comparing GTX 1080 vs RX 480? HOw unfair can you be. Compare to GTX 1060!

    This isn't a GPU comparison. The article's scope is to show what VRScore is all about.

    That is irrelevant, it would be much better seerved by including a GTX 1060 as well.

    I would have liked to, but there are two problems with that.
    I didn't have any more time to run more benchmarks. You have to run each pass 5 times per HMD. More GPUs would mean less HMDs. I cover VR, so the HMDs were my priority.

    The other problem, and this one's the kicker, is that I don't have a GTX 1060. In fact, the two GPUs that I used are the only two current generation cards that I have access to.
    Our GPU reviewers receive the graphics card samples, most of which go to Igor in Germany.
    Reply
  • thor220
    If you are trying to appear objective you are doing a poor job of it

    "Gigabyte's GTX 1080 G1 Gaming has no trouble maintaining roughly 90 FPS with the Rift and Vive, but PowerColor's RX 480 struggled to keep up."

    It should be thoroughly noted in this article that the RX 480 is not AMD's top end offering and that the GTX 1080 and RX 480 are in a different price class. Yet we have here you giving AMD the business for something that should have been obvious from the onset. While you do note that the RX 480 and GTX 1080 are in a different class you only do so in regular font in a text swamp. Something that important should at least be in bold.

    I could also nitpick how the Nvidia card is always above the AMD one or that it's charts come before the AMD ones in the 2nd half, which effects presence of mind. Likely you are projecting your own preference in this instance, perhaps without even knowing.
    Reply
  • rhysiam
    Thanks for the article, it will be interesting to see how these new benchmarks come together.

    To those annoyed by the RX 480 vs 1080 comparison, I think you're completely missing the point. The article clearly states in the title that's it's testing HMDs and trialling a new benchmarking suite. On the methodology page there's a short and very clear paragraph stating that a future article comparing GPUs is in the works (looking forward to that, btw). And it's very clearly and explicitly stated that the goal of this article is NOT testing GPUs. Then, in exploring the results of the benchmark (necessary to see how the new benchmark suite works) we see a GTX 1080 performing better than an RX 480. So what!? 480s start at less than 1/3rd the price. Maybe Nvidia fans should get angry because the 480 achieves ~70% of the 1080s performance at 30% of the price? Or perhaps you could blame the author because they're perpetuating the perception that Nvidia cards are overpriced by only demonstrating the high end (and arguably overpriced) model?

    No, this is an interesting introduction to a new benchmark, clearly labelled as such, and IMHO a worthwhile read. To be blunt, if someone just looks at a few of the charts without reading the article and concludes that Nvidia > AMD, then that's entirely their own fault.
    Reply
  • WFang
    For a review that purports to be about the HMD's, I found the lack of for example photon delay data for each HMD glaring.

    Also, this test tool overall seems to miss the mark for what I was hoping for. This does not appear to be very helpful when it comes to comparing HMD's which in my opinion is what we need more. Figuring out what hardware works best is important, but can be approximated by looking at e.g. 4k game tests already.

    As for the differences in render vs display resolution, it would be nice if it was possible to force rendering to the display native resolution and/or the lowest resolution HMD in the test round-up.
    Reply
  • bit_user
    I'm surprised they didn't just use USB 3, for the light sensor module.

    19238012 said:
    As for the differences in render vs display resolution, it would be nice if it was possible to force rendering to the display native resolution
    Until they address this, it seems the tool is useless at characterizing what a user of that HMD would actually experience.

    Apparently, 6 months' delay wasn't enough. It's still not ready for use.

    That said, I look forward to downloading it for the eye candy.
    Reply
  • sleven
    Running HDK 1.3 at 2880x1620 pixels seems to be a major handicap in this test. An overfill of 1.5x from the native 1080p resolution seems like an arbitrary setting, but I guess it's used because it was on "by default" depending on how the server was installed. Setting "renderOverfillFactor" to 1.0 would run at the HDK 1.3's native resolution of 1920x1080. 1.5x overfill is overkill for HDK.
    Reply
  • sleven
    HDK 1.4* is interchangeable with HDK 1.3 in this case
    Reply