Leap Motion Merges AR And VR So You Can See Your Real Hands

For the last couple of years, ever since Oculus' successful Kickstarter campaign, many people have had the opportunity to try out virtual reality for themselves. Often, one of the first questions after putting on a VR HMD is "Where are my hands?" The feeling of being in a digital world loses much of its "realness" once your brain realizes your hands aren't visible.

Leap Motion was one of the first companies to recognize this disconnect and quickly refocused its plans of bringing gesture control to VR. Early iterations of such controls were interesting, yet clunky. Most users who tried the Leap Motion controller in conjunction with VR didn't feel the experience was all that good. The digitized hands being shown in the scene never really felt like they were your own.

Now, Leap Motion released a new version of the Unity Core Assets bundle used to develop for Leap. The latest version brings with it support for a new feature called Image Hands, which uses aspects of both augmented reality (AR) and virtual reality (VR). Using the Leap Motion camera, the software projects raw images of your own hands on top of the 3D hands already known to Leap Motion users.  

By projecting the raw image of your hands onto a 3D mesh, the Image Hands software is able to bring your real hands into the virtual environment. With the use of the twin Leap Motion cameras, the software is able to use both images of your hands to track depth correctly. Hands can now be seen disappearing behind 3D objects, for example.

Leap claimed that the use of this hybrid infrared/dual camera system to track hands reduces jitter effects. Dropping the artificially generated hands out of the equation made a big difference. Hands are represented on the screen accurately even when passing through other objects.

This has been used as an advantage. Leap has included a function where the onscreen hands glow to help guide users to form proper hand gestures. A similar function was included to indicate interaction with objects. When an item is touched in the real world, the onscreen hands light up. This is a simple substitute for the lack of haptic feedback from the virtual environment.

All the information you need to use Image Hands in your own project is available here, and you can download the core assets pack from here. If developing for Leap isn't your thing, but you'd like to give it a try, ElementL: Ghost Story is a free game that has support for the new hands.

Follow us @tomshardware, on Facebook and on Google+.