HTC Vive Cosmos Tested: Benchmarks and Performance
SteamVR or Viveport? HTC, Make up Your Mind!
The Cosmos hardware isn’t perfect, but for the most part it's an improvement over previous Vive headsets. The software, on the other hand, leaves a lot to be desired. When Vive first introduced the Viveport platform, it pitched it as an improved marketplace that would offer a better experience for consumers and higher revenue share for developers—presumably a win-win situation for everyone. However, the execution and delivery of Viveport hasn't lived up to the goal.
That was never a problem before because the Vive and Vive Pro headsets were developed first for SteamVR, and Viveport was built around the headsets. The Cosmos is the first PC-connected headset that Vive designed for Viveport first, SteamVR second, but the Cosmos still requires SteamVR to function. During the setup process, you must either log into an existing Steam account or create a new one and install Steam. To access Viveport, you press the Vive button on the right controller, and to access SteamVR, you double-tap the Vive button on the left controller.
Viveport is supposed to be easier to use than SteamVR’s software library and marketplace, but when you first put the headset on, you’re thrust into the SteamVR Home environment, not Viveport. It's confusing and, frankly, hindered my ability to navigate either platform effectively.
What’s worse, Viveport doesn’t to do a better job at curating titles that work with the Cosmos. I am writing this ahead of the consumer launch of the headset, so some developers may not yet be prepared with Cosmos-ready builds of their titles. That said, not even all the games marked as “Top-50 for Cosmos” worked with my Cosmos.
After spending the weekend fumbling around trying to navigate between SteamVR and Viveport content, I would much rather use SteamVR alone rather than a combination of the two content delivery systems. Sadly, that does not appear to be possible.
The biggest problem that I encountered while testing the Cosmos is the controllers’ compatibility with games. Many of the games that I tried to play simply would not work properly because the buttons would not respond to my inputs.
For instance, Arizona Sunshine has been part of our performance testing suite ever since we’ve done VR performance testing. However, I was unable to test how the game runs on the Cosmos because the controllers' grip buttons wouldn't register, which left me unable to progress beyond the initial menu environment because I couldn’t pick up the tapes that launch the settings and games scenarios.
Skyworld is another game where I encountered trouble. The right controller registered fine, but for some reason, the face buttons on the left controller wouldn't work. Many of the game’s functions worked fine with the left controller, but since I couldn't get the left controller to work, I couldn't pull up cards during battle. As a result, I also had to drop this game from the test lineup.
At first, I assumed the problem was that I was playing SteamVR versions of these games instead of Viveport versions. However, I ran into the same problems with the Viveport copies that I later installed. The buttons on the Cosmos controllers, while achieving physical parity with competing controllers, rely on different software mapping. HTC Vive confirmed to me that developers will need to update their games before the Cosmos is fully compatible with existing titles and that updates will roll out "shortly."
Despite the trouble with some games, other titles worked just fine, including Space Pirate Trainer, Beat Saber and Serious Sam VR: The Last Hope. Oddly, the controller model that displayed in Serious Sam was that of a standard Vive wand, but it had no ill effect on gameplay.
Despite trouble getting many games to run properly, I managed to capture performance metrics from a handful of games. Audica, Space Pirate Trainer, Serious Sam and Beat Saber are the test suite for this review.
For performance testing, I used a desktop computer equipped with an Intel Core i7-8700K CPU, 16GB of DDR4-3666 RAM and an Nvidia GeForce RTX 2080 Founders Edition graphics card. I also ran tests with a Nvidia GeForce GTX 970 graphics card to get an idea of the low bar for performance.
For the RTX 2080 tests, I locked in SteamVR’s render quality at 120%, which worked out to 2208 x 2604 pixels per eye. The GTX 970 was unable to perform at 120%, so I allowed SteamVR to set the recommended resolution, which ended up being 44% render quality.
From a graphics performance standpoint, Audica does not push the limits of your graphics card. With both graphics cards, our system had no trouble keeping the framerate above the 90 frames per second (fps) threshold when you stay within the recommended render quality settings.
Space Pirate Trainer
Space Pirate Trainer is an old mainstay in our performance testing suite. The game doesn’t really tax your graphics card at all. In fact, in the past I've managed to play Space Pirate Trainer with integrated graphics on a Windows Mixed Reality headset.
I thought that the game may run at 120% render scale on the GTX 970, but that proved to be incorrect. With the render scale set that high, the GPU barely delivered 65 fps, and it dropped roughly 1,000 frames out of a 6,000-frame test.
With the render settings configured appropriately, both GPUs delivered 90 fps with no appreciable dip in performance.
Serious Sam VR: The Last Hope
Serious Sam VR: The Last Hope ran very smooth, and I found no problems with the recommended render settings.
Both the RTX 2080 and GTX 970 delivered a consistent 90 fps with fewer than 5 dropped frames per session.
Beat Saber is surprisingly hard on GPUs. By default, I had the in-game render scale set to 1.7x over SteamVR’s recommended settings. This proved to be too much for the test system, and I was forced to lower the render scale to achieve good performance.
With the high render setting, my RTX 2080 delivered a framerate of 45 fps. During these tests, the GPU dropped nearly 50% of all frames attempted.
With the in-game render scale turned down, both GPUs handled the task well at their respective resolutions. The RTX 2080 managed to top 154 fps unconstrained. Meanwhile, the GTX 970 achieved 114 fps, which is pretty high and suggests there’s some more wiggle room to increase the graphics quality.
The HTC Vive Cosmos isn't a perfect piece of hardware, but what is? Yes, the headset has a few shortcomings, but it’s still a pretty great device. Going back and forth from the Cosmos to the HTC Vive Pro, Valve Index and Oculus Rift S gave me further appreciation for the Cosmos.
The Vive Pro is a well-built piece of tech but feels rickety in comparison to the Cosmos. I’ve tested the best VR headsets, and I can honestly say that I’ve yet to try on another headset that feels as high-quality as the Cosmos.
I have reservations about the Cosmos controllers' ergonomics, and I wish that more games worked with the controllers upon release (although this will likely change). Plus, there's no lens relief adjustment. However, those are pretty small concerns.
As I said, no headset is perfect. However, the shortcomings of the Cosmos are worth looking past because the rest of the package is quite impressive and a notable improvement over the original HTC Vive.
Image Credits: Tom's Hardware
MORE: Best Virtual Reality Headsets
MORE: All Virtual Reality Content
MORE: Virtual Reality Basics
Any comment about black levels and screen door compared to the index?
Your average frame time graphs didn't list unit of measurement. I assume it's ms? (11.1ms = 90fps)
I'd say it's not particularly attractive pricing, especially for anyone considering it as an upgrade over the original Vive, since they will have base stations already, making the pricing of Valve's Index more reasonable. I wouldn't exactly say the Index is priced to move at $1000 for the full kit, but $500 for the headset upgrade alone or $750 for the headset with the superior Index controllers seems at least somewhat more palatable than $800 for the Cosmos with some janky controllers with poor battery life. I don't have first-hand experience with either headset, but based on this review compared to prior Index reviews, it sounds like the Index is superior in almost every way. Better field of view, higher refresh rate, a more accurate tracking system, and more advanced and ergonomic controllers with several times the battery life and a simpler charging procedure.
About the only notable thing the Cosmos has going for it is the relative ease of moving it from one location to another without the need for setting up additional tracking hardware. But unless you're constantly moving it between systems, that's probably not much of a concern. The price of the full kit is technically a bit lower too, but for the vast majority of people who haven't already got into VR, I kind of doubt they will be looking at a headset in this price range anyway. It seems more likely that they would be looking at one of the many headsets priced around half as much, like the $400 Oculus Rift S, or a "first-gen" headset, or an even less expensive Windows Mixed Reality headset. Functionality-wise, it seems a bit questionable to even consider this a true "second generation" headset. Exactly what new features does it bring to the table? The resolution is about the same as what Windows MR headsets were doing for half the price a couple years ago, and the controllers are not as good as the Oculus Touch controllers from nearly three years ago.
And what kind of review was this that doesn't dive into the quality of the experience, outside of ergonomics and software compatibility at launch? It feels more like a first-impressions piece. How does the image quality compare with competing headsets, or audio quality? Almost nothing was said about these things. Measuring performance seems kind of pointless if you arbitrarily adjust the render scale to make the performance useable on all the hardware you are testing. At least go into detail about how that affects visuals. There seems to be no mention of how a 44% render scale on the GTX 970 makes things look. Probably pretty bad, is my guess. You might as well get the original Vive for a few-hundred dollars less if you are going to do that.