Audio Visual Experience And Other Features
Visual Experience
Although we already covered some of the visual differences between the three headsets earlier when we talked about display tech, lenses and FoV, now we want to compare their overall visuals. Again, since we haven’t been able to use PlayStation VR as much as the other two HMDs, we won’t be able to comment on it in as much detail.
The image above is a composite of two screenshots taken by Reddit user FlyingWaffleED in Elite Dangerous from this discussion. As you can see, the quality from both PC-based headsets is comparable, though the Rift does look slightly better, especially the text. This comparison is not entirely fair, though, because Elite’s developers have admitted that there are some issues with the way text is rendered on the Vive version of the game. But it’s a start, and it shows how close these two systems are visually.
From our examinations of other games, we still think the Rift has a bit of an edge, and this is due to a few factors. As mentioned previously, its hybrid lenses are better at diffusing the screen-door effect. Also, the narrower FoV and higher PPI (pixels per inch) resulting from its smaller displays means that even though both the Vive and Rift run at the same resolution, the Rift’s pixels are visibly a little smaller, resulting in a crisper image. Of course, the difference isn’t going to be that noticeable when you are in VR unless you are deliberately looking for it, and the Vive still has the advantage of a more immersive, bigger field of view and has no issues with light leaking in.
The PlayStation VR is at a disadvantage despite its better subpixel matrix screen technology, excellent optics and good FoV. Its resolution is still much lower. And yet it comes close. Again, once you’re in VR, it might not matter as much.
All three headsets have a good level of brightness, with the Vive being brightest to our eyes. HTC implemented something it calls “Mura Correction” starting with the Vive Pre (the second-generation developer version). This produces a more uniform level of brightness over the display and removes what HTC termed a “linen-like” quality to the images being displayed, which we tried to simulate in the image above. We weren't aware of the issue when we tried earlier versions of the Vive, but when it was pointed out to us, we immediately noticed a big improvement. From what we understand, this was never an issue that needed addressing on the Rift, likely because Oculus implemented something similar from the start.
Refresh Rate, Latency And Frame Rate
We talked up front about what Oculus Chief Scientist Michael Abrash thought was needed, and what would be possible by 2015 to produce a compelling VR experience with a good level of presence. So far we’ve covered optics, FoV and resolution, but we haven't touched on the necessary 90Hz refresh rate and corresponding 90 FPS required for comfortable VR. All three headsets can achieve this, but the methods for doing so differ.
Of course, the easiest way to ensure a constant 90 FPS input to the HMD is brute force. If your CPU and GPU are fast enough then there's nothing to worry about. That's easier said than done though, and for the PC-based headsets it's important to keep the barrier to entry as low as possible. It'd be unreasonable to expect every customer to buy a GeForce GTX 980 Ti or Radeon R9 Fury X. And what about PSVR? Sony’s headset is driven by the substantially less powerful PlayStation 4 console.
Game developers need to ensure that the experiences they create can run at 90 FPS on the recommended configurations set forth by both HTC and Oculus. In a perfect world, that'd be the end of it. But we all know that a host of other factors affect performance even if you do have a GeForce GTX 970 or Radeon R9 290. There are driver issues, Windows issues and other background apps gobbling up available resources.
There is also a misconception that, if we're able to drive normal PC games at 90 FPS using the right system components (especially CPU, GPU and memory), even at high resolutions, VR games should perform just as well, even if we're projecting dual FHD resolution images (forgetting that even those images have to be perfectly in sync). VR performance, which is to say the sense of real presence it delivers, is dependent on latency, but unlike most PC games, VR introduces the need to incorporate precise head movement into the latency equation. That is, what you expect to see isn't just dependent on what you do with keyboard/mouse/controller, but where your head is positioned. Without that last part, presence is lost, and you get nausea. Processing and matching head movement to high quality visual output, and doing so with precision is more taxing on the system.
There are a few techniques built into the VR processing pipeline to assist in conquering this challenge. It's worth noting that, despite these techniques, our own experiences trying to play many VR games with less-than recommended hardware has been pretty unpleasant, and often unplayable.
HMDs work around marginal performance scenarios using a technique called reprojection. Each system approaches this in a different way. In VR, because your view is typically not static, missing a frame (or more) results in what is termed judder, which you can see simulated in the image above. Not only does this impact the image quality but it lowers your visual comfort, and significant judder contributes to VR nausea.
Reprojection counters this by inserting a duplicate of the previous frame into the rendering pipeline to maintain the frame rate, but this frame is adjusted (or warped) to compensate for any change in camera (really the HMD) position from the previous frame, reducing or eliminating judder even if the frame rate drops. The technique Oculus uses for the Rift is called Asynchronous Timewarp (ATW) and it is considered to be the best way to deal with this issue. It is asynchronous because it runs out of sync with the main rendering process and can pre-emptively know if a frame is going to be dropped (by missing the 11ms maximum rendering time needed for 90 FPS) and inserts a warped-for-movement frame in its place. ATW is one significant advantage that the Rift has over the Vive and PSVR.
The only problem with ATW is that, because of its pre-emptive nature, in order for it to work properly there needs to be support at the driver and operating system level. Oculus had to work with Microsoft, Nvidia and AMD to “change OS GPU scheduling, GPU command processor microcode, and GPU kernel driver design to enable ATW.” Because of this, ATW was enabled just before the Rift's launch and only works on a small selection of GPUs.
As a result of those hurdles, Valve isn't using ATW with SteamVR games (which is what the HTC Vive runs). Instead, SteamVR uses an alternative technology that isn’t pre-emptive and works on less powerful GPUs. Valve’s Alex Vlachos explained SteamVR’s interleaved reprojection technique during his GDC 2016 presentation (you can see the slides here). Unlike ATW, it isn’t always on and only kicks in when then VR application drops below 90 FPS.
That means there may be a moment when frames are dropped, which could induce judder. In fact, although it hasn't been an issue for us, we’ve read reports that in certain Vive games, especially graphically demanding ones like Elite Dangerous that need more than the minimum system requirements to run well, judder can sometimes be an issue.
PlayStation VR has it even worse in that the platform's underlying hardware will probably have a hard time hitting a consistent 90 FPS at all, let alone the 120 you'd need to take advantage of its 120Hz display. Sony's solution is to turn on reprojection all of the time, and the minimum frame rate target developers have been told to achieve without it is a more forgiving 60 FPS. At a recent presentation to developers at GDC, Sony went over what it requires to certify a game for use on PSVR and a minimum of 60 FPS is on that list. It is then reprojected to 120Hz.
The other options available are 90 FPS reprojected to 90Hz and 120 FPS reprojected to 120Hz. We’d bet that the first option is what most developers will aim for, and because of this judder shouldn't be an issue on PSVR. The PS4 simply isn’t capable of outputting super-detailed graphics at the resolution and frame rate required to not use reprojection, so we can be sure that PlayStation VR’s visual experience will be a noticeable step down from the Vive and Rift.
But because Sony has a lot more control over what content is certified to appear on its platform, it can reject experiences that do not meet the minimum frame rate and comfort requirements for VR. The more open nature of PC gaming makes it more likely that you end up with a poor-running game, and there's nothing Oculus or HTC can do.
It's also worth noting that performance can often come down to game development. In speaking with developers and gaming engine providers behind the scenes leading up to the launch of these first-generation HMDs, we learned that many of the visual bells and whistles have been toned down. As with so many other aspects of VR, most of us will hardly notice, or care. Developers can also take advantage of techniques like foveated rendering, or Nvidia's newly announced simultaneous multi-projection pipeline, announced alongside the company's consumer Pascal GPU. We suspect more API-level features will continue to enhance the VR gaming experience.
Audio Experience
High-quality audio is just as important as the visuals for an immersive VR experience. Of the three HMDs, only the Rift comes with integrated sound, while the HTC and Sony setups include a set of ear buds (and the option to use your own headphones). All three headsets also have built-in microphones for in-game communication.
The Vive includes a set of HTC earbuds that plug into the ~50cm audio extension cable running out of the headset. So that you don’t have to worry about where to put a tangled mess of cabling, the bundled earbuds have a nice short 30cm cord. They don't look like much, but they're surprisingly good, producing very high-quality audio. We suspect they're a modified version of HTC’s $80 Pro Studio Earphones. The one problem is that these earbuds aren’t exactly easy to share (for hygiene reasons). You can still use your own set of cans if you'd prefer an over-the-ear solution, or you can go with something wireless. In the SteamVR control panel, you can even set the Vive’s audio to go through a different device, such as your PC speakers.
At first glance, the Rift's integrated audio doesn’t seem very good. But looks are deceiving; Oculus paid just as much attention to the design of those little speakers as it did with other aspects of the Rift. Not only did the company prioritize great sound, but it also wanted the headphones to move out of the way as you put the HMD on and take it off, and then be easy to adjust as you use them.
One big advantage for game developers is that the Rift's audio subsystem present a single spacial profile to tune for, which means everyone who plays it will experience the same source. If you don’t like the Rift’s headphones, you can remove them with an included tool. But keep in mind there is no headphone jack or break-out audio cable available, so you need to run your own headphones all the way back to your PC. If you are determined to use something other than what Oculus gives you, wireless is likely the way to go.
PlayStation VR is like the Vive in that there is a headphone jack to use your own audio setup, along with bundled earbuds. We haven’t had a chance to use them though, and can’t comment on their quality. The Processing Unit box that comes with PSVR is also used for 3D audio, offloading it from the PS4’s host processor to free up cycles.
All three headsets have built-in microphones for in-game chat, though the Vive's leaves a lot to be desired. We’re not sure if it’s a software issue that can be fixed with firmware or a problem with the mic hardware itself. Thankfully, you can change the audio recording source in SteamVR to a different device, allowing you to connect a better microphone. You can pair the Vive to your phone through Bluetooth and use the mic to take phone calls while you're in VR. Because that's a pretty useful feature, we hope HTC sorts out the Vive’s mic issues soon.
The Rift’s microphone is a lot better; it doesn’t have us wishing for another solution. Again, as with the PlayStation VR’s included headphones, we haven’t been able to test Sony’s microphone yet.
Extra Features
HTC's Vive is the only HMD with a front-facing camera. It's such a useful feature that we wish the other two had it as well. It was rumored that the Rift would come with a camera (there were some images of a prototype with one that ended up on Oculus' site), but for some reason it was dropped from the final consumer version. If there's one aspect of the Vive that makes it more compelling than the Rift, this, and what it enables, is it.
Part of the Vive’s room-scale VR experience is SteamVR’s “Chaperone” system, which determines the boundaries of your play space (you configure this when setting up the Vive) and warns you in VR when you are close to the edge of the playable area by layering a grid-like visual barrier. This safety feature is to prevent you from potentially walking into a wall or furniture with the Vive on your head. What the camera adds is the ability to see objects outside the play area for added safety; you can see the “Chaperone Room View” screenshot below.
While the camera enhances the Vive’s safety system, it also lets you interact with real-world objects while in VR, something we dubbed “Tron mode” when we first saw it (shown in the “Room View” screenshot above). A button on the controller transforms your view in VR to a neon blue (or any other color, since you can change it) filtered picture from the camera, letting you sit in a chair, have a drink or simply find a controller you put down without removing the HMD.
Another feature of the camera is the preview mode, which is turned on in the SteamVR control panel. With it active, a small video window is attached to the side of the controller so you see where you are while accessing the SteamVR interface. HTC also suggests that the camera software could be upgraded to have object recognition, so the Vive could detect hazards like a pet running into your VR play space.
We mentioned in the audio section that you can pair your Android or iOS phone to the Vive, and once you do, you can receive calls, texts and calendar notifications while in VR. At the moment, a small pop-up window appears, notifying you of a call or text. But your interactions are currently limited. You can answer calls and receive texts, but can only send canned replies from within the in-VR Vive Home application. You can also reply to texts with a call, but there is no way to dial out to initiate a new call or compose a new text, and the canned text replies only work if you have an Android smartphone.
The last extra feature on the Vive is its USB port, used to attach accessories like a Leap Motion sensor (though to use Leap with the accessory port it is recommended that you turn off the Vive’s camera). However, the extension cable included with the Pre developer Vive isn’t included, so you’ll likely need to acquire one to use accessories with the consumer Vive.
With the USB accessory port being an integral part of the Oculus Rift DK2 (also commonly used with Leap Motion) it's odd that the Rift CV1 does not have an accessory port. In fact, the Rift doesn’t have any extra features beyond its core VR functionality.
Despite the fact that the PlayStation VR does not have a camera, an accessory port or any phone integration (at the moment; it could be a feature coming to PSVR by the time it launches), it does have one big bonus that sets it apart from the Vive and Rift: its Processing Unit. Along with 3D audio acceleration, it displays what Sony calls the “Social Screen” to your TV while using PSVR. There are two modes for this display. The first, called mirroring mode, is an undistorted, cropped and scaled view from the right eye of the headset. The second, separate mode can show a totally different image from what you see in VR.
Unlike PC gaming, console gaming usually happens in front of the TV in more social situations, and Sony wants to promote that VR doesn’t have to be an isolating experience. With mirroring mode, your friends can easily see what you are doing in VR, and although they can’t interact with you, at least they can get an idea of what you are experiencing. The separate mode can take social VR a step further by enabling asynchronous multiplayer games. One player can be in VR viewing the game from one perspective, and up to four other players can be using regular PS4 controllers to interact with the VR player. This one feature is something that sets PlayStation VR apart from the Vive and Rift.
