Samsung HMD Odyssey Performance Analysis
To run our tests in Gorn, we entered the arena and selected two weapons to fight with. After saluting to start the fight, we pressed the Scroll Lock key to start FCAT VR and capture our gameplay. We set FCAT VR to give us a 5-second delay so that we could get back into position after starting the performance monitoring.
Gorn is a Unity-based game, and it runs very well on our standard test rig. With the GTX 980 Ti, we observed no performance issues. And we didn’t observe a significant performance deficit caused by the higher-resolution panels in the Odyssey headset. The extra resolution resulted in roughly 1ms of additional latency.
When we switched to the GTX 970, the performance dipped somewhat on paper, but the in-headset experience offered no perceptible change. Strangely, we observed better performance with the Odyssey headset than with the Rift headset on the lower-end GPU. The GPU dropped more frames when pushing pixels to the Odyssey, but overall the latency was lower on the Odyssey HMD. We would have expected the opposite result, because Gorn doesn’t officially support the Windows Mixed Reality headsets on SteamVR.
Rick & Morty: Virtual Rickality
Owlchemy Lab’s Rick & Morty: Virtual Rickality is another Unity-based game. Unlike Gorn, this title actually offers official Windows Mixed Reality support, which suggests that the developer optimized the title for Windows MR headsets.
For the Rick & Morty test, we fired up stage 6, which is the level in which you get a pair of hand blasters and are tasked with shooting government agents out of the sky. We chose this scenario because we believe it’s the most action-packed scene in the game.
Our GTX 980 Ti-equipped test rig had no trouble powering the Odyssey headset in Rick and Morty. The extra pixels bumped the latency slightly, but at no point did we see higher than 7ms motion-to-photon latency.
When we swapped out our GTX 980 Ti for a GTX 970, the story changed, though. We observed significant frame-rate loss when powering the Odyssey with a GTX 970. However, our test system still managed to deliver almost every frame within the 13ms threshold.
Arizona Sunshine from Vertigo Games and Jaywalker Interactive is another Unity-based game, but the graphics fidelity in this title is significantly better than in Gorn and Rick & Morty. In fact, Arizona Sunshine offers advanced graphics features that require a high-end, multi-threaded processor to drive.
For our tests, we set the in-game graphics to their maximum adjustment and enabled the advanced particles and environment destruction options. Arizona Sunshine includes a campaign mode and a never-ending Horde mode. We opted to test Hord mode so that we could repeat the same test over and over. We chose the Canyon level, which puts us inside a shack waiting for zombies to come after us
Arizona Sunshine is more demanding than Gorn and Rick & Morty, but our GTX 980 Ti had no trouble driving the experience on all three headsets. In fact, the performance differences among the three devices were negligible, with less than 1ms deviation among the three.
When we switched to the GTX 970, we noticed a dip in performance, but we still observed enjoyable performance across the board. If you’re trying to power a Samsung Odyssey HMD with a GTX 970, you may want to dial back the graphics settings a little bit. As you can see in the graph, the GTX 970 was flirting with the 13ms barrier, which would have resulted in significant reprojection to maintain the frame rate.
Serious Sam VR: The Last Hope
Serious Sam VR: The Last Hope was Croteam’s first crack at a VR game. Since its release, the developer ported its back catalog to VR, but The Last Hope is Croteam’s sole VR-only title.
Unlike most VR games, Serious Sam VR: The Last Hope features an Options menu with adjustable graphics settings to help you dial in the best possible performance. We wanted to maintain consistency between tests, so we manually adjusted our pre-set values. Instead of fine tuning each option, we set the CPU to high, GPU to medium, and memory speed to high, and allowed the game to fill in the rest automatically. We repeated that process for both GPUs and verified that the settings didn’t change between HMD switches.
Serious Sam VR: The Last Hope offers a campaign mode and a never-ending wave mode. We performed our evaluation in the never-ending wave mode, which enabled us to repeat the tests as many times as we need. Each time you play the wave mode, the order of enemies that you face changes, but the density of enemies increases at roughly the same pace every round.
As expected, our GTX 980 Ti delivered a smooth, tear-free experience for all three HMDs. However, we noticed that the latency slowly crept up from under 4ms to above 5ms. Curiously, the Vive test peaked at more than 6ms of latency. These low figures don’t translate to a noticeable difference in-game, but it’s interesting to see how much the number of enemies affects the performance on screen.
The GTX 970 didn’t fare so well in Serious Sam. This was the first game in which we noticed the difference while playing the game. We observed significant frame drops from the headsets with higher-resolution targets. The GTX 970 didn’t have any trouble with the Rift’s resolution, but the card started dropping frames near the end of our test with the Vive headset. And the Odyssey HMD pushed the GPU hard enough to make the experience choppy and unsettling. You would need to dial the graphics settings back to get a playable frame rate, particularly with Samsung's headset.
Island 359, developed by Cloudgate Studio, is the only title in our grouping that was built on Epic Game’s Unreal Engine 4, and it’s the most demanding title that we tested with the Odyssey headset. Island 359 features an open-world jungle to explore and a variety of dinosaurs to hunt, including Raptors, Stegosaurus, and T-Rex. The game also offers a third-person spectator mode called Virtual Self, which makes it a great game to share with friends. We didn’t enable Virtual Self because it takes a toll on performance and we didn’t want to tax the GPU unnecessarily.
For our Island 359 tests, we started a new Mercenary mode mission in the jungle map. Once we gathered our gear and set foot into the bush, we started our 60-second benchmark run. We did our best to follow the same path each time, but it's easy to get lost in the jungle environment.
As you would imagine, our GTX 980 Ti once again handled the workload with ease. While we were playing the game, we couldn’t tell the performance difference between the three HMDs, but we were surprised by the results when we loaded the data into FCAT Analyzer. Somehow, the Samsung Odyssey, with its higher resolution and higher rendering target, observed the best performance. The results from the Vive and the Odyssey were neck-and-neck, with a slight edge for the Odyssey. The Rift, however, ran at a serious performance deficit, with the latency hovering around 13ms for the duration of the test.
Of course, the GTX 970 fared worse in this test. The game ran fine on our HTC Vive headset, with a few dropped frames, but nothing serious enough to hinder the gameplay experience. The GTX 970 couldn’t keep up with the demand of the Oculus Rift or the Samsung Odyssey, though.
The Oculus Rift hovered in the mid-20ms range, which translated to constant reprojection. The data indicate that the GTX 970 didn’t deliver any frames with less than 13ms of latency, which means our GPU was not producing anywhere near 90 frames per second.
The GTX 970 couldn’t deliver many on-time frames to the Odyssey, either. The data there indicate that the GPU dropped nearly every second frame.
We spoke with Steve Bowler, CEO of Cloudgate Studio, about the Rift performance problem, and he suggested that it may be a result of the Oculus Runtime running in the background. Cloudgate is aware that Rift users experience performance degradation, and the company is searching for a solution.
MORE: Best Virtual Reality Headsets
MORE: All Virtual Reality Content
MORE: Virtual Reality Basics
Why? Is it the inferior Vive controllers? The sweet last-gen outside-in tracking? Displays? The extra $100 for the DAS? When you make a comment like that, it's nice to point out why.
I believe there was a fairly glaring omission from your otherwise thorough review.
WMR requires Bluetooth 4.0. As BT is more prevalent on laptops than desktops, that might be a gotcha when people get their WMR unit, regardless of manufacturer & model.
"Cool! It's finally arrived!".... "Wait, WTF...I need a BT adapter??? GAHHH!"
Here's a couple of tips that I didn't note in my own review from my month+ spent with the Odyssey.
1) Before anyone purchases any WMR device, you should run Microsoft's WMR PC Check tool. It's a free download in MS' App Store. (HERE) If there's any "X" that it calls out as a problem, WMR will not work. Period.
2) This is the BT adapter that MS officially(?) endorses on their website. Link goes to product on Amazon.
3) I found the controller tracking to be very inconsistent with the Odyssey. In your gaming/testing, did you do much movement where the controllers needed to go to the outer stretches of your arms or behind your head (for games such as Space Pirate Trainer and Beat The Blitz)? My controllers with fully juiced batteries would lose sync which really detracts from the total immersion.
Also, you are correct about the battery cover. As I noted in my review, my grip on the controller was dislodging the battery cover.
4) If my understanding is correct, the controllers pair to your BT adapter. This potentially causes signal attenuation if your BT adapter is plugged into a rear USB port or is simply not always in line-of-sight with the controllers.
5) I too found the tether to be too short. So I ended up buying some extension cables which resolved the problem. Be aware, not all extension cables are created equal. For example, a 'standard' USB extension cable does not infer USB 3.0 compatibility. And WMR is very specific if you aren't plugged into a a USB 3.0 port. I thought maybe it was just power delivery, but I was initially using a powered USB 2.0 extension cord. No go. It has to be a USB 3.0-rated extension cable. Same thing with HDMI certifications.
So here's what I recommend for people looking to extend their playing area. (Obviously, I confirm these work with the WMR headsets, specifically the Odyssey)
- 6' HDMI extension cable- USB 3.0 extension cable- Any run of the mill USB 2.0 extension cable so get your BT adapter closer to the play area.
6) Lastly, with regards to the comparison of built-in audio on the Odyssey compared to the lack-of on the Vive... The Vive has the 1/8" headphone tap on it. A solid set of earbuds fill the need nicely. Their cable drapes over the back of your head and are never in the way. So people don't have to resort to placing big, over-the-ear headphones over the Vive's head strap. I will say that the AKGs on the Odyssey do work exceptionally well!
Again, a nice thorough review. I just wanted to help plug what I consider to be a few holes so the readers can avoid a lot of the delays and headaches that I initially dealt with.
And I also agree- especially given the Vive's price drop since the Vive Pro released- the Vive is still the overall HMD winner.