Hands-On With HTC's Vive Tracker

Minutes after HTC revealed the Vive Tracker at its press conference at CES 2017, the company opened up numerous demo stations to showcase the device's potential. We set out to do as many demos as possible during our limited time in the showcase area. A majority of the demos featured games that showed off the capabilities of the tracker, but there were also a few stations that showed off different potential uses for a physical VR object.

Say Cheese

The Vive’s main attraction is also its best window into the future of VR. Games (or, more precisely, demos) are what attracted people to VR. So it makes sense that games would lead the charge again in showing off the tracker.

One of those games was Exoplanet from a studio called dotdotdash. It tasks you with using a special camera that can process images with different filters, like X-ray or infrared, to learn more about the alien species in front of you. With the Vive controllers, each stick acts like your in-game hands and the camera is just another virtual object within the game. With the Vive Tracker, however, the camera gains a physical form that can actually be held.

The developers at dotdotdash utilized the tracker, which some developers called “The Puck,” in addition to creating their own version of a sci-fi camera to make the concept work in VR. “It’s super straightforward,” said lead interactive developer Zach Krausnick. “You literally just calibrate to the puck, and then once you have this puck calibrated in the virtual space you just put your model on it.”

“The interesting about the whole process [is that] we use the same 3D model that we use to fabricate the physical device that we’re actually using in the virtual space,” Krausnick said. “...literally in the virtual environment, we can make the model do anything we want to [such as] the spectrum analysis for the sound, the overlay for the image that you’re seeing through the viewfinder… we can get creative as we want with that."

The result is impressive, but more importantly, it’s natural. I used the camera's many filters, which could be switched between using a button on the camera, to identify the alien. Once the strange creature revealed its true form, I pressed another button to snap a photo. The camera itself performed as expected: Latency was barely noticeable and the experience of holding a virtual object through its real-world counterpart was astounding. In the past, creators could only mimic an object if it was placed on the Vive controller, but with the new tracker, developers can craft any model they want onto the device to make their games more realistic.

Guns Blazing

Firearms will probably be one of the most popular peripherals for VR. Two demos at the Vive event featured guns; each utilized the Vive Tracker in a different way. A studio called Master of Shapes offered our first gun-toting demo and its game, Cover Me, utilize the tracker in a multiplayer VR game.

Cover Me is a first-person shooter where you stand in a small, enclosed area while you shoot down hordes of enemies coming towards you. In VR, you still use the standard Vive controller as the gun. A friend or two can join you in the game, but they don't need to do so via VR. They can just see the action on their phone.

The process works with a mobile app of the game in addition to a network connection. (The phone and the Vive must be connected to the same network in order for it to work properly). The phone is then mounted on a toy gun (the one used at the event was bought online) which acts as the game's controls. To create the feeling of camaraderie, the gun muzzle is plugged with the Vive tracker. This allows you, the VR player, to see your friends (or rather, their gun) in the virtual space.

Even though the app was slow and buggy at times--it’s still in development, as is the VR version--the gameplay was exciting. I had to constantly walk around the play area to see if enemies were coming. After all, they could sneak up behind me due to my limited field of view on the phone. At the same time, though, one of the developers helped me out in VR so I wouldn’t have to go through the level alone. The gun worked well with the phone. Every trigger pull registered on the screen as a shot, and gunshot sounds emanated from the phone to lend yet another layer of immersion to the experience.

There are some multiplayer games out there for VR (Hover Junkers is a popular example) but the fact that you don’t need multiple VR HMDs to play Cover Me with friends is a new idea. You can still show off the powers of VR to your friends, but this game allows them to join in on the fun instead of simply observing your actions on a screen.

The Evolution Of Arcades

There was another shooter I played after Cover Me called ROM: Extraction. Similar to the former game, ROM: Extraction also utilized a firearm, but this time around it was an assault rifle with the tracker placed where the scope would be fitted. This demo was even more immersive with the fact that I put on a small backpack that provided haptic feedback whenever I opened fire.

The gameplay reminded me of Raw Data from Survios. I was on an elevator platform surrounded by enemy robots and I had to take them out while simultaneously dodge enemy fire. When my assault rifle wasn’t enough, I could pull down on the buttons near the rifle’s front grip to launch a grenade that exploded on contact. Needless to say, the short demo was action-packed with enemies swarming you from all sides. It was also exhausting because I had to keep moving around and crouching to avoid bullets. The gun itself felt realistic, not just because of its looks, but also because of its weight. It didn’t feel like some plastic toy. It was somewhat heavy, and keeping it propped up to fire at robots became cumbersome near the end of the demo. At one point, I just kept it at waist-level and launched grenades.

The developers set up a green screen area around me to show off a mixed reality feed of what I was seeing in the game. But don’t expect to have the same experience at home. When I spoke to the developers at First Contact Entertainment, they told me that the gun and vest came from a company called VRsenal, which specializes in providing VR experiences for the military as well as arcades, some of which already have a few booths dedicated to VR. The mixed reality portion, then, would benefit those at the arcade as a form of souvenir.

Still, the added peripherals definitely made the game more exciting. With the tracker’s size and light weight, peripheral makers such as VRsenal don’t have to worry about making a device that would accommodate the Vive controller. Instead, they can focus on making an accurate, realistic version of a gun and place the tracker wherever they want and still provide a solid, immersive experience.

Track Your Hands

Even with a game or experience without peripherals, the tracker can benefit developers, especially those who are focused on hand tracking. At the event, both Noitom and Manus VR showed off their hand-tracking capabilities with the new device.

Both companies seemed to incorporate the same idea by putting the tracker on the glove. Again, this solves an issue that plagued the Vive controller, which was too large and slightly heavier. The new device is smaller, so you can put it on the tracking glove without it interfering with any hand motions.

In both cases, the tracker does as its name suggests: tracks the position of each hand. That leaves the company to work on the more intricate part of their devices, mainly the nine-axis inertial measurement unit that captures the orientation of each finger for more accuracy.

However, the demos for each device was different. Manus provided a tech demo that showed not just the hands, but also arm tracking, which was a bit surreal, especially when it was over. As I wrote down notes, I had this strange, out-of-body feeling that I can only assume was a result of the accuracy of Manus’ tracking technology.

As for Noitom, I tried its open-source demo, which allowed me to play with numerous objects such as balls and dominoes as well as the ability to paint. In my opinion, this was far more impressive, because I was amazed by how easily I could control a ball’s movement or the speed at which I hit a row of dominoes. The main attraction in the demo was painting, because while I created strokes of paint with one hand, I could adjust the stroke’s thickness with the other hand by opening or closing the gap between my thumb and index finger. The quick response in the change of brushstroke thickness was astounding, especially when I drew a line and repeatedly changed the thickness as it appeared in the virtual space.

In the end, however, the focus was on the tracker, which provided a smaller and lighter alternative to the Vive controller to find the hands’ position in VR. This frees up the developers of hand-tracking games to utilize the controllers or even additional tracking devices for numerous peripherals similar to my experience with Exoplanet or ROM: Extraction.

Possibilities

The small event showed different ways to utilize the tracker across numerous scenarios, but this is just a small sampling of what is possible with the tracker. More companies can use it to create peripherals of the realistic or fantasy variety, others might use it to enhance their current VR game, or hand-tracking companies like Noitom and Manus see it as a better alternative for “seeing” the hand in VR.

In the end, the CES 2017 event was a taste in what’s to come this year and beyond. Up until now, Vive experiences were limited to the controllers that came with the box. With the addition of the tracker, developers can experiment with new ways for players to interact with a virtual space, which could result in more immersive experiences that further blur the line between the real and virtual worlds.