How We Tested The Samsung HMD Odyssey
The Samsung Odyssey is the third Windows Mixed Reality HMD that we’ve had the chance to evaluate, but it’s the first one that we’ve been able to run objective benchmarks on. When we received the Acer Windows MR headset last year, we were unable to run Nvidia’s FCAT VR, and we were unable to use Basemark’s VRScore on the new HMD. We spoke with Nvidia and Basemark at the time, and both companies confirmed that their tools didn’t work with Microsoft’s VR platform. Nvidia told us that FCAT VR would eventually support Windows MR. Basemark, however, said that it didn’t expect to support Windows MR because Microsoft didn’t seem interested in helping Basemark get the benchmark working.
We were losing hope for benchmarking this platform as a whole. Nvidia took FCAT VR off its website in December, and it hasn’t yet released a new version, and Basemark has not yet released an updated build for VRScore. Faced with a limited-time evaluation loan, we once again had to resign to subjective evaluation.
Fortunately, we received a new beta copy of FCAT VR from Nvidia just in time to test Samsung’s headset. The new build of FCAT VR doesn’t officially support Windows Mixed Reality, so we can’t test Windows Store content. However, Windows Mixed Reality headsets are compatible with Valve’s OpenVR API, which FCAT VR does support. The results may not be 100% accurate, but we should be able to get a reasonable approximation of how well games run at the Odyssey’s 2880x1600 resolution.
Test Setup
For this evaluation, we used our Intel X99 testbench, which features an Intel Core i7-5930K clocked at 3.5GHz, a Gigabyte GeForce GTX 980 Ti Extreme Gaming, and four sticks of 4GB of Crucial Ballistix DDR4 2400 RAM. We also ran tests with an EVGA GeForce GTX 970 SC to see how well the entry-level VR-ready GPU can power the high-resolution Odyssey HMD.
| CPU | |
| Graphics Card | |
| Motherboard | |
| Memory | Crucial Ballistix Sport DDR4-3200 (4x4GB) |
| Hard Drive | Crucial MX200 500GB |
| Power Supply |
We also pulled out our Oculus Rift and HTC Vive for comparison, so that we can get an idea of how much of a performance hit the higher-resolution displays make you take.
Our Testbench features Windows 10 Professional, and at the time of the evaluation, we were running version 1709, OS Build 16299.309. We used Nvidia's GeForce Game Ready Driver 391.01 for our game test.
SteamVR Configuration
SteamVR has a feature called the rendering target, which is the resolution that the GPU renders the scene before sending it to the VR HMD. The rendering target is generally higher than the actual resolution of the displays, which is supposed to increase the clarity somewhat. The rendering target setting is independent of the HMD type and isn’t dictated by the resolution of the panels in the headset.
The Oculus Rift and HTC Vive both include dual 1,080x1,200 panels, and yet SteamVR’s rendering target for the two devices is 1344x1600 and 1512x1680, respectively. Steam VR treats the Samsung Odyssey, with its higher-resolution displays, to an even higher 1433x1772 pixels.
SteamVR features a slider that enables you to increase or decrease the rendering target for each HMD, which technically enables us to push the same pixel count to each device. But instead, we opted to leave the setting at the base value of 1.0 for each headset because we believe that most people would not tweak these settings before playing. We’ll explore the performance overhead of supersampling adjustments in a future article.
Test Software
We selected five games to test for this evaluation, which vary in graphics fidelity and complexity: Gorn, Rick & Morty: Virtual Rickality, Arizona Sunshine, Serious Sam, and Island 359. These five titles also represent three different game engines: Unity 3D, Serious Engine, and Unreal Engine 4.
Gorn and Rick & Morty don’t offer adjustable graphics settings, so we ran those games as is. We left the preview pane in their default windowed configurations. Arizona Sunshine, Serious Sam, and Island 359 offer adjustable graphics settings, and we normalized the parameters for each headset and GPU configuration.
For Arizona Sunshine, we set each video setting to the maximum value, including draw distance and texture quality. Serious Sam VR features pre-set graphics configurations, which are dictated by the hardware in your system. We hit the auto-detect button with the GTX 980 Ti installed, which set the CPU speed to ultra, GPU speed to medium, and the memory clock to ultra. Each time we changed a variable, we set the configuration back to the original auto-detected settings. In Island 359, we set the graphics to the high preset, which sets every parameter to high and the draw distance to 130.
For each test, we captured the framerate data during 60 seconds of gameplay, then compared the results in Nvidia’s FCAT VR Analyzer. We ran each test multiple times to rule out anomalous behavior.
MORE: Best Virtual Reality Headsets
MORE: All Virtual Reality Content
MORE: Virtual Reality Basics
