Acer Windows Mixed Reality HMD Review

How We (Couldn't) Test The Acer Windows MR HMD

When we received the Acer Windows Mixed Reality Headset, we hoped to run some benchmarks on it. Previously, we dabbled with Nvidia’s FCAT VR utility, which reports graphics performance using Oculus' Rift and HTC's Vive. Unfortunately, FCAT VR does not support Microsoft’s Windows Mixed Reality platform, though. We did reach out to Nvidia regarding WMR testing and were told that FCAT VR wouldn't be updated in time for the launch.

Although we were disappointed by Nvidia’s response, we weren't surprised. It took the company more than one year to develop the first iteration of FCAT VR, and constant runtime updates have necessitated maintenance releases ever since. We have full confidence that Nvidia will eventually add WMR support to FCAT VR, but until then, we're unable to quantify frame rates or dropped frames as we did in FCAT VR: GPU And CPU Performance in Virtual Reality.

As of December 7, 2017, Nvidia pulled the FCAT VR download from its site and left a note stating that a new version would be available soon. If the upcoming build supports Windows MR, you can be sure that we'll test with it.

Display Performance Testing

FCAT VR is a great tool for evaluating the VR graphics pipeline's performance, but a host PC’s ability to deliver high frame rates doesn't matter if the headset's displays can't keep up. Last year, Basemark released a utility that measures how fast an HMD’s display reacts to the signal coming from your computer.

Basemark’s VRScore software pairs with a device called a VRTrek, which features two photodiodes that detect light emitting from your headset’s displays. The software determines the latency between a command's initiation and its execution on the display. In the past, we used VRScore and the VRTrek device to evaluate the performance of Oculus' DK2, Oculus' Rift, HTC's Vive Pre, HTC's Vive, and the OSVR headsets. Unfortunately, we may never be able to run the same test on Windows MR headsets.

Before we received Acer's solution, we checked with Basemark to see if there would be a new version of VRScore with support for Microsoft’s Mixed Reality platform. The company planned to release VRScore 1.1 over the summer, but was forced to delay its release for several months. We still don't know when the new version will be available. But even when Basemark gets 1.1 out the door, it still won't include Windows MR support.

“Regarding Microsoft VR headsets, we had to, unfortunately, cancel the support as nor we or our current partners were able to get enough interest from Microsoft to help us out with the integration,” said Arto Ruotsalainen, Basemark Co-Founder, and CEO. “VRScore PC 1.1 will have updated Oculus, HTC Vive, and OSVR SDKs.”

With that news, our hands are tied. Lacking the tools to test real performance, we can’t verify Microsoft’s claims. Subjective experience is all we can offer at this time. We hope that, eventually, performance analysis won't be as challenging. But until then, we're at the mercy of an emerging market.

Subjective Tests

We tested Acer's Windows MR headset with three different systems. First, we used our primary VR test system, which includes an Intel Core i7-5930K, 16GB of Crucial Ballistix Sport DDR4-3200, and a Zotac GeForce GTX 980 Ti AMP! Extreme. Then we tried the headset with less powerful machines, including a system with an Intel Core i3-4330, 8GB of G.Skill DDR3-1600, and a GeForce GTX 950. Microsoft also made a big deal about Windows MR headsets working with the HD Graphics engines in Intel sixth-, seventh-, and eighth-gen Core processors, so we auditioned an Intel Core i5-6500 and 16GB of Corsair DDR4-3200.

First, we tried Acer's Windows MR headset on our highest-end system. This allowed us to experience what a premium experience should look like. Backed by a powerful host machine, Microsoft’s Windows Mixed Reality platform ran well. Its frame rate felt smooth, and the visuals were crisp. Our informal test sequence suffered no perceivable dropped frames, leaving us satisfied.

We did, however, experience frame rate problems when we enabled a preview window to let others see the display output. Microsoft’s spectator view contributes significant load, bringing performance down across the board. When we have access to a version of FCAT VR that works with Windows MR, we’ll quantify the spectator window's true impact.

Next, we moved on to a lower-end Core i3-4330 and GeForce GTX 950. This system exceeds Microsoft’s minimum requirement for Windows MR, but it doesn’t satisfy the baseline for an Oculus Rift or HTC Vive headset. To our surprise, though, it had no trouble with mixed reality. In fact, we had a hard time discerning between the Core i7- and Core i3-based PCs. We played around in the Beach House, pinned apps on various walls, and jumped between windows to push the hardware as hard as possible. We even fired up Bigscreen, which calls for a Core i5 and GeForce GTX 970 at least, and noticed no performance hitches.

VR Through Integrated Graphics

Finally, we tried Acer's Windows Mixed Reality Headset hooked up to a Core i5 with Intel HD Graphics. Microsoft made a big deal about the Window MR platform’s ability to run well without discrete graphics, and we were skeptical of its ambitions from day one. Nevertheless, we were eager to test the company's claims.

To accommodate the lower performance of an integrated graphics engine, Windows Mixed Reality headsets must operate at two refresh rates. When a discrete GPU is present, they refresh at 90 Hz. Without an add-in card, they run at 60 Hz.

Oculus and Valve put a lot of time and effort into determining what makes VR comfortable, and both companies concluded that 90 frames per second is the sweet spot where most people experience no negative side effects. When the frame rate dips much below that threshold, users start to get sick. Thus, we weren't sure how Microsoft's pursuit of 60 Hz would go, and our real-world experience didn't help. In fact, we don't think 60 Hz should be an option.

Of course, Microsoft’s intent is to make immersive technology accessible to a wider audience, which should drive adoption for MR (and VR) hardware/software. However, if mixed reality leaves you with a bad taste in your mouth, you probably won't line up to buy it. And unfortunately, the most accessible configuration for Windows Mixed Reality proves to be incredibly jarring. By now we’re accustomed to VR, and we’re rarely affected by motion sickness. But Windows MR running on integrated graphics ranks among the worst experiences we’ve ever had.

As you might expect, graphics quality in the Beach House is reduced when you don’t have a discrete GPU installed. The environmental details aren't what worry us, though. Performance is the more serious issue. As soon as we turned around for the first time, we wondered what Microsoft was thinking. As we rotated, the scene had trouble keeping up. Intel's HD Graphics engine couldn't even maintain a consistent 60 FPS. The problems worsened when we tried moving around. Pressing the joystick to teleport, we noticed a prominent delay between controller input and on-screen action. There was even lag when we moved our hand to place the teleport reticle in a different place.

The primary purpose of supporting Windows MR headsets on integrated graphics hardware is to make the platform appealing in the business world. This idea falls flat due to the performance implications, though. Sitting in front of a static window works fine. Turn your head to the side to check another screen, though, and the image gets choppy, quickly becoming uncomfortable.

Gaming powered by integrated graphics was the only redeeming experience, ironically enough. i-illusions somehow managed to smooth out the performance of Space Pirate Trainer on the most basic hardware configuration. Of course, all of the advanced graphics details were disabled, but we still had to be impressed by the fluid frame rates. Few games demonstrate this level of refinement, and fewer support the entry-level Windows MR hardware configuration. But if you can find a few gems like Space Pirate Trainer, you could potentially justify using Intel’s HD Graphics as a temporary solution. Still, we do not advocate skipping a dedicated GPU for long-term use.

MORE: Best Virtual Reality Headsets

MORE: All Virtual Reality Content

MORE: Virtual Reality Basics