We were sufficiently excited to see what Oculus has cooking with Project Santa Cruz, its untethered Rift prototype with inside-out roomscale mobile tracking, but just the night before, we saw the same clever tech--using regular smartphone cameras and software-based sensor fusion to enable 6DoF tracking--employed by a company called Dacuda.
All You Need Is A Smartphone
In a hotel lobby across the street from Oculus Connect 3 (which was in the San Jose Convention Center), we met with Dacuda’s Erik Fonseka and Lukas Schleuniger. Fonseka pulled out his smartphone and showed us real-time, roomscale 6DoF tracking on it. What was surprising about this demo was the fact that he was using just his smartphone. It had no extra sensors strapped to it, and there were no external markers at play.
Dacuda’s inside-out mobile tracking software is called SLAM Scan, and it employs a technique known as sensor fusion. The software uses data from the smartphone’s existing camera and onboard sensors (such as the gyro and accelerometer) to calculate where the device is in space and translates the physical location and movement into a movement within a virtual environment.
This is a big deal because almost all mobile VR solutions currently on the market--including the excellent Samsung Gear VR and Google’s impending Daydream--have only 3DoF tracking. That is, you can enjoy your virtual experience by strapping on a headset and sitting quietly while you look around. Essentially, 3DoF tracks only your head movements; if you stand up and walk around, the virtual world will not move with you. To get that positional tracking, you need 6DoF.
One mobile XR product that does offer 6DoF is Google’s Project Tango. Shipping soon via Lenovo’s Phab2 Pro smartphone, Project Tango offers a world of (primarily) augmented reality capabilities, but it requires special hardware, such as a motion camera, structured-light projector, and an IR camera.
Oculus’ Project Santa Cruz also uses standard smartphone cameras (although it uses four of them to Dacuda’s one) and software sensor fusion. But instead of being nervous about this sudden, Facebook-funded competition, Dacuda was glad to see it.
“It really confirms what we are doing in the space of inside-out tracking and how relevant this is for roomscale VR,” Dacuda’s Lukas Schleuniger told Tom’s Hardware later in an email. “This is fundamentally different from the Oculus approach, which is doing the inside-out tracking with a separate and specialized hardware headset, which results in a much higher price.”
In other words, yes, it’s essentially the same tech, but Oculus is aiming for a highly proprietary, tightly-integrated implementation. By contrast, Dacuda’s SLAM Scan tech can be implemented widely, on essentially any smartphone (iOS, Android, and even specialized hardware) that has the requisite onboard sensors. And because those smartphones serve as the engine and display of most mobile VR--working in tandem with “dumb” HMDs for viewing, such as the aforementioned Gear VR and Daydream--Dacuda has an extremely wide and growing potential install base.
“Our goal is to bring true room-scale VR to as many people as possible, and for that you need a low price point and a broad range of platforms,” said Schleuniger. He further noted that because new smartphones bring better displays and technology every year--in other words, merely because of existing and predictable market forces--the Dacuda-based experience would continue to improve.
Things are moving fast for Dacuda. Just last week, it suddenly forged a collaboration with MindMaze to get a version of its technology into the company’s healthcare-oriented “MMI” platform. Next week, Dacuda CEO Dr. Peter Weigand will be at Qualcomm’s 4G/5G Summit in Hong Kong, presenting a talk about how software can turn standard smartphones into 3D scanners.
I agree with you. Granted... i have VERY limited programming knowledge/experience but I feel like this would have been somewhat simple to do awhile ago.
Can anyone who knows jump in and explain what is complicated about this? Or potentially details on how it works?
The basic concept is obvious, but tricky to get working well enough to be usable without barcode posters on your walls, ceiling, and maybe even the floor, considering the poor FOV on a cellphone camera.
Getting it to work on a phone while VR apps are running would also be a challenge. Phones already overheat and have poor battery life when using them for VR. Add more processing requirements, and that gets worse. Phones will get faster and better at dissipating heat, but games will also get more demanding at a similar rate.
As far as heating I'd bet you could make a buck or two selling a passive heatsink back shell/cover for VR use. Especially if this take on mobile VR is as good as it seems to be and takes off.
Next year should be a good year for mobile VR....
It's true that you need a good smartphone, but the value there is that almost everyone *already* has one. So buying the phone is not really part of the expense. (Maybe you'd be lured into buying a more expensive phone than you would otherwise, though.)
With Rift/Vive, you need a burly PC and THEN buy the $800 VR components. With this sort of tech, all you need is an inexpensive HMD. Cardboard is just a few bucks.
You raise a good point about dedicated VR hardware versus something that does double duty. In the next year or so, we're going to see how well that concept plays out. Google's Daydream is built on that concept, so at least one ginormous company is wagering that this sort of lower-end VR is going to make people happy and engaged, at least to an extent.
I think what Dacuda isn't telling you is that the environment needs a certain amount of visual clutter. If you're standing in a small room with bare plain walls, then you'll need to hang some posters. This is why Oculus, MS, ... everyone else needs multiple cameras. They at least need to be able to see things like multiple corners, where the walls meet the ceiling or floor, in order to reliably extract pose information accurate enough for AR or VR.
IMO, it's not surprising they're running this on a phone. The questions are: what is the platform requirement and how much of the CPU/GPU is left for your app? The mid-range/low-end phones have been improving at a pretty good pace, so most 64-bit phones will probably have adequate performance. As Jeff says, battery life & heat will be issues, if the app depends on their SLAM engine continuing to run at full frame-rate, for very long.