State Of Tango, Google I/O 2017

Tango is potentially Google’s biggest gamble. A complex package of cameras and depth sensing cameras designed around augmented reality, Tango presents some unique challenges for Google.

The search giant is no stranger to ambitious moonshots, but unlike its Daydream View VR headset, Google Glass, or even Google Home, all of which were designed and sold by Google, Tango relies on Android partners to invest in the plan.

When Apple wants to add a major new feature or remove a headphone jack, all it has to do is build a new iPhone. By contrast, Android is open source, and Google relies on partners to manufacture Android phones (some partners, including many Chinese OEMs and Samsung, don’t always seem to want much to do with Google). Google has not yet manufactured any Tango devices itself; it’s relying on partner companies to voluntarily manufacture phones that include these large, complicated, expensive camera setups.

So far, only two phones, from Lenovo and Asus, include Tango, but Google clearly expects it to be a part of every Android phone in the not too distant future. This will take some serious work to happen, but the potential payoff could be huge.

Mixed Signals

If you haven’t heard of it, Tango is google’s augmented reality platform. It uses a combination of cameras and depth sensors (lasers, generally) to detect and track the geometry of whatever you point the camera at in order to create effects and apps that blend the virtual and the real. Last year at I/O, Google’s developer conference, only a few apps were on display.

When we first saw a Tango device in 2016, it was a bulky camera bar built into a custom tablet. You could use it to measure a table leg; scan a chair to make a 3D model of it; look at a virtual dinosaur that seemed to be standing in the room with you; or play Woorld, a game from the creator of Katamari Damacy, where you interact with a number of friendly critters who wander around whatever room you are in.

They were all interesting in their own way but didn’t exactly signal a revolution.

But developers are just starting to experiment with the potential applications, and from what we saw this year at I/O, Tango is picking up steam.

An important caveat here: The tech demos we saw were mostly shown via video or on-stage presentation, or were in tightly controlled environments. It’s hard to properly evaluate any new app or piece of hardware until you get your hands on it and try it out in less than ideal situations. That said, here are just a few of the things potential uses for Tango.

Putting Tango on the Map

At its most basic level, the Tango sensors let you create a 3D map of a room. Extending this idea out, developers have come up with a number of potential use cases for a good spatial map. You can easily create a 2D floor plan of your house or apartment by simply walking through the rooms and pointing your phone’s camera around.

Another demo showed a scan of a room where at the click of a button, all the furniture was edited out, which leads to another potential use case--using the AR sensors to artificially place furniture in your artificially empty simulation of a room. It’s perfect for virtual redecorating, or seeing if your couch will fit comfortably in a new apartment before you move.

Image 1 of 3

It’s worth noting that the Asus Zenfone AR, the second phone to hit the market with a Tango sensor (coming this summer), is also Daydream ready, meaning you can use it to make a full color 3D map of a space and then pop the phone into a Daydream VR headset and do a virtual walkthrough.

The ability to accurately map a room also leads to an idea that’s a little less obvious. By comparing the camera input with a spatial map (which can be created ahead of time and shared or downloaded), Tango can provide centimeter-level accurate positioning and navigation, even without GPS.

This is actually similar to how the “Worldsense” tracking on the new Qualcomm-designed standalone VR HMD is supposed to function. The camera maps tracking points (generally high contrast points, edges, textures, etc.) and uses their relative position to determine their location in space. Combine this with the depth sensors on a Tango phone, and you have extremely accurate local positioning. Going one step further, Tango’s ability to overlay virtual objects on the physical world means your phone could show you around a city or a store by drawing an arrow on the ground in front of you (Deadspace, anyone?)

Image 1 of 3

An example of this is the AR museum exhibits Google has developed with Tango. In the Singapore ArtScience museum, for example, visitors given Tango phones could see the museum virtually transformed into a rainforest and follow markers on the ground to exhibits on virtual animals throughout the space. (This is similar to the demo we saw in Barcelona over a year ago.) Another exhibit of an Egyptian mummy used a Tango device to show a 3D CAT scan of the mummy as you moved the device over the exhibit. The Detroit Institute of Arts is using Tango, as well.

Life in the Real World

It’s always interesting to look at the industrial potential of new consumer tech, a place where Google actually excels but gets relatively little press. Google Glass never took off with consumers, but it was apparently quite popular with some surgeons and engineers for its ability to display readable text or images while their hands were occupied.

Similarly, Google mentioned an example use case for Tango where a mechanic could point a phone at a malfunctioning engine, and an expert back at the office could highlight problem parts by drawing on the screen. Through Tango’s recently improved “anti-drift” algorithms, the highlight stays pinned to the part in question, even if the mechanic moves the phone away.

Engineers continue to improve the system, too.  In addition to the anti-drift updates, new improvements to Tango allow for better occlusion (moving VR content behind actual objects) and environmental interaction. In one simple yet fairly amazing demo, a virtual pool ball convincingly reflected back an actual piece of red paper as it was moved around the ball.

This year finally shows that there are practical reasons a consumer could want a Tango device. The barriers to success are still huge, but if Google can convince consumers, manufacturers, and developers of the possible advantages, AR may become as common as fingerprint readers on the smartphones of the future.