Benchmark Results
Job Simulator
What better place to start testing than one of the titles that has been with us through the hardware's development? Job Simulator was one of the first room-scale experiences shown to the press and public, and it will soon be in the hands of everyone who purchases a Vive. Valve and HTC partnered with Owlchemy Labs to offer the game as bundled title.
To test, we opened up the Office Worker scenario and ran through the same sequence of activities over a two-minute period, making sure to have plenty of head movement to represent what you’ll likely be doing inside the game. There doesn’t appear to be any way to manipulate the graphics settings, so we ran the game as it is.
Job Simulator is very polished, and it presents stable performance, although there are occasional spikes in the frame times.
Curiously, the R9 390X and GTX 970 appeared to encounter significant performance dips. But upon closer examination it turned out that the R9 390X dropped to 47 FPS for a moment and shot right back up to 90 FPS. The GTX 970 hit a similar snag, dipping to 54 FPS within the first five seconds of the test. It raced back to 90 FPS over the next three seconds.
Tilt Brush
Tilt Brush is another title that comes bundled with the hardware. It isn't so much a game as it is an art application. Enthusiasts will spend hours in Tilt Brush, and performance will be key for longer sessions.
Within Tilt Brush, there is a gallery that features pre-drawn art. When you open these images up, the drawing plays out as an animation in front of you. We simply chose the same scene (Sarah Northway’s Space Dragon) and watched it get drawn while walking the same path for each benchmark.
Tilt Brush demonstrated the most consistent frame rates of the five applications we tested. This wasn’t a surprise though, as there's not a lot going on graphically. The test isn't particularly taxing; rather, it's included to demonstrate that you can use lower-end hardware for certain VR applications. Even the R9 285 offers an enjoyable experience.
Space Pirate Trainer
Spice Pirate Trainer was chosen because it is one of the most active games you can play right now. It’s a first-person gallery shooter that requires you to duck and dodge away from oncoming laser fire while you shoot at flying droids coming from all directions. The action requires that you look around rapidly, which really taxes the GPU.
The benchmark run is simple: just play the game. Space Pirate Trainer restarts at the beginning every round, so for each run we simply recorded data for the first two minutes of game play. Rounds tend to only last a few minutes anyway.
Space Pirate Trainer proved to be taxing on many GPUs. The Radeon R9 285, R9 380X and GeForce GTX 960 fared abysmally in this game. For each of those cards, the droids flying around looked like double-vision as they went by. It was very uncomfortable, and I wouldn’t recommend playing the game with such low performance. The GeForce GTX 780 wasn't even fast enough to deliver consistent frame rates.
The Gallery – Episode 1: Call of the Starseed
The Gallery offers one of the deeper narratives that you’ll find in an early VR game. Cloudhead Games has been working on this title for years, and its hard work is apparent. The environment is rich and the movement system (Blink) that the team created makes traveling through large environments a breeze. This game has plenty of visual flare to take in, and the audio production is impressive. It'll undoubtedly be a popular choice for folks looking for more than just a quick burst of fun.
Our benchmark involves running through the first two minutes of the game over and over. You start off in a cave and make your way to a beach where you discover an abandoned campsite with a burning fire and some loose fireworks. To test the lighting system, we fire off one of the pyrotechnics toward a rock face, which illuminates the scene. Once the firework burns out, we proceed through the caves to the first clue.
The Gallery proved to be the most demanding selection from Valve's list of launch titles. On all of the GPUs that satisfy the platform's recommended hardware list, game play is subjectively smooth. But the same can't be said for the lower-end cards. The GTX 960 and R9 380X were playable, but just barely. Enthusiasts looking to enjoy their first VR experience won't be completely happy with this. And the R9 285 was bad to the point that I had to keep closing my eyes to get through the run. Nvidia's GeForce GTX 780 is by far the worst card in this title. Despite an average frame rate of 70, the spikes up and dips down create a prominent flicker effect.
Vanishing Realms
Vanishing Realms came out of nowhere, but this one is going to impress early adopters. The game was first revealed during this year's GDC, though there was nary an image or video to preview the project. I haven’t had chance to dive deep into the content yet, but the first few minutes hit the right chords.
You play as a hero in an action-adventure RPG (think Link from the Zelda series, only in first-person). You get a sword. You can smash boxes and pottery to collect golden coins and urns. Oh, and you get to fight life-sized sword-wielding skeletons.
For the benchmark run, we started a new save and ran through the first two minutes of the game, following the same path at the same pace each time.
Actually playing through the game, my experience was comfortable and smooth. But when I had a chance to parse the data and look at the frame time graphs, it became clear that a lot of the frames took longer than the 11.1ms corresponding to 90 FPS.