Test Setup and Conclusion
Our test system has evolved since the last time we looked at the 5K Plus. In our pre-production review, we powered the headset with a system equipped with an Intel Core i7-5930K. Some readers questioned if our CPU choice caused poor performance results, so we updated our VR test system.
Our new machine is equipped with an Intel Core i7-8700K overclocked to 4.7Ghz. It also includes 16GB of G.Skill Sniper X DDR4-3600 RAM and an Asus Prime Z370-A motherboard. Our games and OS are installed on our 500GB Crucial MX200 SATA SSD. We used an Nvidia GeForce RTX 2080 Founders Edition to drive the pixels on Pimax’s dual-1440p ultrawide headset. Instead of testing a variety of GPUs, we opted for a variety of screen resolutions.
The game selection is somewhat different than the previous analysis with, several games overlapping. Arizona Sunshine, Space Pirate Trainer and Serious Sam VR: The Last Hope returned, and we replaced Beat Saber and The Gallery: Heart of the Emberstone with Doom VFR and Project Cars 2.
When Pimax initially revealed the 5K Plus, the company said it would offer a 200-degree FOV. However, during development, Pimax realized it needed narrower options for less-powerful computers. Pimax’s software package includes an option to limit the FOV to increase performance. The Large setting enables the full width of the displays to give you a 200-degree (diagonal) viewport. The Normal setting reduces the FOV to 170 degrees, and the Small setting brings it down to 150 degrees. The narrower options disable the pixels at the outer edges of the screen to lower the workload for your GPU.
In theory, narrower FOV settings should result in better performance. However, as you’ll see from our results, that isn’t necessarily true. In our experience, the Normal FOV option is the most demanding. But we wouldn’t be surprised if a future update corrects this aberration.
For the purpose of this evaluation, we adjusted the SteamVR render scale settings to change the resolution of the delivered image. We left the render scale settings in the Pimax PiTool software at 1.0 because changing it introduces too many variables for a review of this scope. However, we tested each game at multiple resolutions in each of the three FOV configurations for each of our five games.
|PiTool FOV setting||Large||Normal||Small|
For the Small FOV setting, SteamVR recommended a render target of 68 percent, which equates to 2298x2714 pixels. With the Normal FOV, Steam recommended 48 percent, which works out to 2669x2280 rendered pixels. And the Large FOV setting defaulted to 26 percent, a 3543x1678 resolution.
Space Pirate Trainer
Space Pirate Trainer is one of the games we used when we looked at the 5K Plus last year. In our pre-release evaluation, our test system struggled to deliver acceptable frame rates. Even with an RTX 2080, our test system couldn’t reach an average of 90 fps.
This time around, the game performed much better. In fact, the average frame rate for every resolution and FOV setting hit the 90 fps threshold, except for the Normal FOV setting with SteamVR’s render target set to 100 percent. But even then, the average remained above 87 fps.
Given our performance results, we suspect we could increase the anti-aliasing from its current 4x to 8x and still achieve buttery smooth performance.
Serious Sam VR: The Last Hope
Next, we fired up Serious Sam VR: The Last Hope and played many rounds of the Endless Wave mode. When we did this with the old test machine and the early build of Pimax PiTool, our system rarely hit 90 fps, and our RTX 2080 couldn't top 70 fps at Steam’s recommended resolution. When we published our review of the pre-production model of the 5K Plus, we received criticism for not adjusting the in-game settings to improve performance. While that critique holds merit, we decided that an apples-to-apples comparison is more relevant for determining what improvements Pimax made to its software, so we used the same in-game settings this time.
Somewhat disappointingly, we noted roughly the same level of performance in Serious Sam VR as we did in our pre-production review. With the Normal FOV and 64 percent render scale, the difference in the average frame rate is within 2 fps of our previous tests.
The default settings in Serious Sam VR proved too much for our test system. We recommend dialing back the in-game render scale setting and some of the detail options to boost performance.
Despite the low frame rates, the gameplay felt smooth, and I didn’t have any discomfort at most resolutions. The exception is when I enabled both Normal FOV and 100 percent render scale. The frame rate floated around 40 fps in this configuration, which was not a pleasant experience inside the headset.
Arizona Sunshine is another game that required settings adjustments to achieve consistently smooth frame rates. Only the lowest resolution and FOV option produced a steady 90 fps performance. We used the same settings that we used to review the Vive, Vive Pro and Rift, with the texture quality set to high, render scale and LOD at the default value of 1.0 and the resolution set to high. We also enabled the advanced CPU features.
For most configurations, the frame rate felt smooth and gameplay was not choppy. I didn’t notice any glaring indications that our machine was heavily relying on reprojection technologies. However, the results show otherwise. All three SteamVR-suggested resolutions failed to reach an average of 90 fps, and the configurations with higher-than-proposed resolutions didn’t even reach 70 fps. However, dialing back the graphics settings should easily unlock enough performance to reach the requisite 90 fps threshold.
The Normal FOV setting with 100 percent render scale option achieved the worst performance of the group, with an average frame rate of 51.17 fps, which is more than 15 fps lower than our results with 100 percent render scale on the Wide FOV setting.
Doom VFR is a newcomer to the benchmark suite, so we don’t have an old baseline for comparison. That said, we didn’t adjust the game's image quality settings and instead allowed the game to choose the settings for us. The auto-detect process set the lighting, shadows and particles quality to high. However, the game seems to have a built-in override that prioritizes performance over image quality. Despite the high image quality settings, the image quality degraded as we scaled the resolution up, which enabled reasonably high performance even when our headset was configured for 6949x3291 pixels.
Our test system always maintained an average frame rate of at least 80 fps on this game, and more often than not was just shy of 90 fps. Once again, the Normal FOV option with 100 percent render scaling was the most difficult option we tested. However, the image quality at this setting far outweighed the Full FOV setting with the same render scaling.
Project Cars 2
We don’t often use Project Cars 2 for VR benchmarking because the game has a stationary playing position and is incompatible with motion controllers. However, the 5K Plus’ ultrawide FOV is the perfect solution for racing simulations, because it allows a similar peripheral view as a motorsports helmet. Unlike the rest of the test lineup, Project Cars 2 is first and foremost a flat-screen racing game with VR compatibility added to it. As such, the game has more graphics settings than most VR games.
We set most of the options to conservative levels with anisotropic texture filtering set to 4x, post process AA set to low and most of the detail settings set to medium. Particle level, which was set to high, was the one exception.
Unsurprisingly, Project Cars 2 produced the lowest frame rates of the test suite. None of our configurations came anywhere close to achieving 90 fps averages. All three Small FOV options and the Normal FOV configurations returned averages of 45.45 to 47.46 fps, which suggests the game was automatically throttling the frame rate to achieve a smooth 90 fps through reprojection.
Curiously, when we cranked the FOV value to its maximum setting, the average frame rate increased by nearly 50 percent, with all three resolutions returning upwards of 60 fps averages. And the unconstrained fps averages increased from the low-50s to the mid-70s. We can’t explain why the frame rate increased with the higher FOV, but we triple-checked the results and retested for good measure, so we’re sure of their veracity.
The Frametime results depict remarkable conformity between resolutions, which is perhaps the most puzzling characteristic of these results. We see a slight increase in frametime as the resolution scales up, but the differences are just barely outside the margin of acceptable error.
Despite the significantly under-par frame rate that our test system pumped out, that frame deficit didn't impact our perception of the game's performance as we watched from our headset. The game ran at a smooth 90 fps using reprojection, for which we credit Valve’s constant improvements to the SteamVR platform. Without a robust reprojection algorithm, our experience with the 5K Plus would have been much worse.
Image Quality Abnormalities
The Pimax 5K Plus is capable of producing an incredibly sharp image that enables you to read text and see details that you can’t possibly bring into focus on a Rift or a Vive. When we got the right combination of FOV and render scale, the graphics in our favorite games came to life like we’ve never seen before in VR. However, with the wrong combination of settings, the image quality was quite bad.
In general, when you increase resolution, you see an improvement in clarity and sharpness. However, for some reason that is not the case with the Pimax 5K Plus. We found that the best-quality image was often the lowest output resolution. Despite having the dynamic override disabled, either SteamVR or the games we were testing lowered the texture quality to accommodate the resolution, which made the games look terrible.
Doom VFR was the biggest culprit. At the highest resolution of the Full FOV setting, the texture resolution dropped so low that the zombies looked about as good as they did in the classic Doom games.
Is the 5K Plus Really More Immersive?
VR is all about immersing you in the moment, and Pimax believes that a wider FOV is the key to increasing immersion. We can’t really argue with Pimax on that front. With a wider FOV, you can see more of the virtual world and less of the inside of your headset, which naturally brings you deeper into the virtual environment. There’s no question: the Pimax 5K Plus offers a more-immersive experience than competing VR headsets. Over time, we learned to ignore the black interior walls of our other headsets, but with the 5K Plus we can’t even see the sides unless we manually limit the FOV.
With the Large FOV setting, the screen reached the edges of our peripheral view. In our previous evaluation, the lenses produced significant distortion along the edges. But with the headset's latest firmware, the distortion is no longer perceivable unless you look for it. With the Normal FOV, you can’t see the distortion at all, and the black mask that covers the screen isn’t very noticeable. The black edges were visible in the Small FOV configurations, but you still get a wider FOV with this setting than you do with a Vive or Rift.
And when you get the resolution dialed in correctly, the details you can see in VR games make the experience all the more believable.
Positives about FOV aside, Pimax touts the Pimax 5K Plus as a second-generation VR headset, but we’re still not convinced. In some ways, the 5K Plus takes big steps towards the next generation of VR hardware, but in other ways, it takes big steps backwards.
It is the first ultrawide FOV VR headset available for the consumer market, which is a welcome advancement. But it doesn’t feel like a well-rounded next-gen device. We took a lot of flack for calling the 5K Plus a generation-1.5 device when we reviewed a pre-production sample. But after reviewing a production version, we stand by that conclusion. It's true Pimax is disrupting the status quo by offering a wider FOV and higher resolution than competitors, but the company seems to have ignored everything else about making a great VR headset.
The 5K Plus is sort of like a purpose-built drag car. Unlike muscle cars that can hold their own on a road track, drag cars are built to go fast in a straight line a quarter-mile at a time. They don’t need to go around corners or travel long distances. As such, the designers ignore anything that won’t make the car better at its primary purpose.
After spending a significant amount of time with the headset, I’m convinced that Pimax’s design team approached the 5K Plus with a similar mindset. Pimax laser-focused on delivering ultrawide VR. But in doing so, it ignored important features, such as a mechanical head strap system, built-in headphones and accommodation for people wearing glasses.
The Pimax 5K Plus is not a perfect VR headset, but it’s certainly not a bad one. We wouldn’t trade in a HTC Vive Pro for a 5K Plus, but we would consider it if we were looking for an upgrade from the original HTC Vive.
Want to comment on this story? Let us know what you think in the Tom's Hardware Forums.
MORE: Virtual Reality Basics
Photo Credit: Tom's Hardware