Mobile, Inside-Out Roomscale 6DoF Tracking With Any Decent Smartphone?

We were sufficiently excited to see what Oculus has cooking with Project Santa Cruz, its untethered Rift prototype with inside-out roomscale mobile tracking, but just the night before, we saw the same clever tech--using regular smartphone cameras and software-based sensor fusion to enable 6DoF tracking--employed by a company called Dacuda.

All You Need Is A Smartphone

In a hotel lobby across the street from Oculus Connect 3 (which was in the San Jose Convention Center), we met with Dacuda’s Erik Fonseka and Lukas Schleuniger. Fonseka pulled out his smartphone and showed us real-time, roomscale 6DoF tracking on it. What was surprising about this demo was the fact that he was using just his smartphone. It had no extra sensors strapped to it, and there were no external markers at play.

Dacuda’s inside-out mobile tracking software is called SLAM Scan, and it employs a technique known as sensor fusion. The software uses data from the smartphone’s existing camera and onboard sensors (such as the gyro and accelerometer) to calculate where the device is in space and translates the physical location and movement into a movement within a virtual environment.

Dacuda: Mobile Roomscale Inside-Out 6DoF Tracking On A Regular Smartphone

This is a big deal because almost all mobile VR solutions currently on the market--including the excellent Samsung Gear VR and Google’s impending Daydream--have only 3DoF tracking. That is, you can enjoy your virtual experience by strapping on a headset and sitting quietly while you look around. Essentially, 3DoF tracks only your head movements; if you stand up and walk around, the virtual world will not move with you. To get that positional tracking, you need 6DoF.

One mobile XR product that does offer 6DoF is Google’s Project Tango. Shipping soon via Lenovo’s Phab2 Pro smartphone, Project Tango offers a world of (primarily) augmented reality capabilities, but it requires special hardware, such as a motion camera, structured-light projector, and an IR camera.

Looming Competition?

Oculus’ Project Santa Cruz also uses standard smartphone cameras (although it uses four of them to Dacuda’s one) and software sensor fusion. But instead of being nervous about this sudden, Facebook-funded competition, Dacuda was glad to see it.

“It really confirms what we are doing in the space of inside-out tracking and how relevant this is for roomscale VR,” Dacuda’s Lukas Schleuniger told Tom’s Hardware later in an email. “This is fundamentally different from the Oculus approach, which is doing the inside-out tracking with a separate and specialized hardware headset, which results in a much higher price.”

In other words, yes, it’s essentially the same tech, but Oculus is aiming for a highly proprietary, tightly-integrated implementation. By contrast, Dacuda’s SLAM Scan tech can be implemented widely, on essentially any smartphone (iOS, Android, and even specialized hardware) that has the requisite onboard sensors. And because those smartphones serve as the engine and display of most mobile VR--working in tandem with “dumb” HMDs for viewing, such as the aforementioned Gear VR and Daydream--Dacuda has an extremely wide and growing potential install base.

“Our goal is to bring true room-scale VR to as many people as possible, and for that you need a low price point and a broad range of platforms,” said Schleuniger. He further noted that because new smartphones bring better displays and technology every year--in other words, merely because of existing and predictable market forces--the Dacuda-based experience would continue to improve.

Things are moving fast for Dacuda. Just last week, it suddenly forged a collaboration with MindMaze to get a version of its technology into the company’s healthcare-oriented “MMI” platform. Next week, Dacuda CEO Dr. Peter Weigand will be at Qualcomm’s 4G/5G Summit in Hong Kong, presenting a talk about how software can turn standard smartphones into 3D scanners.

This thread is closed for comments
    Your comment
  • Joao_Pedro
    When they say Oculus will be higher price... probably we will need a $400 to 700 phone to use this application. I wonder how much expensive a dedicated system will be. I am sure will be around the same price or even cheaper with much better performance. Phones are developed with "phone" in mind not to be a VR equipment. But developing and optimizing is way to go, and probably can be a cheap way for developers to have equipment to test.
  • grimfox
    Frankly, I'm surprised it took this long for someone to figure all this out.
  • hdmark
    250841 said:
    Frankly, I'm surprised it took this long for someone to figure all this out.

    I agree with you. Granted... i have VERY limited programming knowledge/experience but I feel like this would have been somewhat simple to do awhile ago.

    Can anyone who knows jump in and explain what is complicated about this? Or potentially details on how it works?