Metaio Demos Exciting AR Future With Mundane Hardware

For as exciting as VR (virtual reality) promises to be (Oculus, HTC/Valve, OSVR, and so on), AR (augmented reality) is perhaps the more practical technology that has more applications for daily work and life. If VR is the golden child, AR is its more practical, less flashy younger brother.

Microsoft's Hololens augmented reality technology has promised the moon, but there are less grandiose uses for AR, as well, as demonstrated by companies such as Metaio.

Metaio's mission is to promote the development of AR. It offers the Metaio SDK for pros, as well as its Metaio Creator software for novices, to enable the creation of AR applications. The company also offers publishing and management tools, so you can deploy and license your AR apps and even track their performance.

Metaio has cast a wide net, targeting marketing, retail applications and publishing; industrial applications; and, most important for many of us, computer vision solutions for semiconductor makers and device OEMs.

Metaio showed us a handful of demos at Mobile World Congress. In one scenario, a Metaio rep pointed a phone at a model car, and we could see a robot inserted into the scene on the phone display. As he moved the phone around, our view of the 3D robot changed commensurately. Basically, the software estimates the pose using the orientation of the camera relative to the scene, establishing X, Y and Z axes.

There was certainly a bit of stuttering, but to be fair, we were watching real-time movements being computed on the iPhone's CPU; there was no special hardware involved. Metaio's software used the company's SLAM (Simultaneous Localization and Mapping) tracking capabilities; according to Metaio, auto maker Audi is already experimenting with SLAM in its auto manuals.

Note, for example, in the image above (still using a stock iPhone), that the Metaio software superimposes headlight beams and a door-and-window design on top of a model Audi car. It's a bit rudimentary, but you can see the beginnings of useful and enticing use cases.

The technology uses 3D tracking algorithms to extract either points or edges from an object (points are easier on objects with flat surfaces, but edges work better for 3D or reflective surfaces). In the phone demo, it's capable of extracting 50 points per frame; running on dedicated hardware, the technology is capable of 500 points per frame.

The software is quite hardware-agnostic and can even use a single camera (like the iPhone used in the demo), although that use case requires a "known" object for scale. For example, one application was an Ikea catalog that allowed you to scan a piece of furniture from the catalog using a custom app and "place" it in your home, at scale, to see if and how and it might fit.

However, the software can also use an IR camera to determine depth. This ostensibly eliminates the need for a given object to be "known," as the camera and software can extract scale automatically.

Image 1 of 2

Another demo Metaio had running in its booth used Intel's RealSense cameras to enhance a shopping experience. An attendant sat at her laptop to shop online for earrings. She was able to select different earrings and see them virtually dangling on her ears, being properly occluded by her head, allowing her to "try them on" before buying anything.

To create this experience, the cameras and software imaged her face and constructed a 3D model for the virtual earrings to interact with. The video image runs on top of this, and the whole multilayered image comes together on a monitor.

This sort of application could work either in-store at a kiosk or at home, for online shoppers who have the appropriate hardware and software on hand. Enhancing the online shopping experience like this offers tremendous promise and could foment competition among e-tailers. (Imagine if Amazon offered such a service but eBay did not.)

This is all computationally intensive, and Metaio said that it's currently working on bringing a chip to market to accelerate this computer vision. We were told that the timetable is 18-24 months.

A final demo, which Metaio was particularly excited about, involved something called Thermal Touch. Simply put, it enables any surface to be a touchable-and-readable surface by using thermal imaging. Wearing an HMD (head mounted display), or simply using your phone's camera and display, you can touch any surface and see that you've left a temporary thermal "fingerprint" that a camera can read.

For example, Metaio showed us a poster on a wall that had covers of several '90s rock albums. Looking through an iPhone display, the rep touched an album and left a thermal fingerprint, which the software recognized and downloaded the album he touched to his phone.

Metaio noted that the demo was rather buggy (and it most certainly was), but it was a proof of concept that made a point. Imagine strolling around a city, seeing a band's concert flyer, and being able to touch that simple piece of paper and download that group's music, check out their website and social media, and more.

What's exciting about this technology is that it allows for rich interaction with AR. In a video, for example, Metaio showed how you could play chess with a pal; each player wore an HMD and saw the same AR chessboard projected onto a table. The players touched the virtual objects to move them, and using Thermal Touch, the actions registered as commands.

The big takeaway here is that it's the software, more than the hardware, that makes for powerful AR experiences. It is true that weak hardware is a limiting factor, but the demos we saw from Metaio used off-the-shelf, commonly-available hardware.

Better, more optimized hardware (such as the aforementioned IR camera or dedicated processor) will help, though, which is why Metaio is helping to develop a better chip. Ideally, that chip could go into phones, HMDs, and other devices to provide better AR experiences.

Keep an eye out for Metaio to become a major player in the AR space. At Mobile World Congress this year, the company's small booth was off in the proverbial woods in Hall 8 (a surprisingly long hike from the press room in Hall 3), but we wouldn't be surprised to see the company in a hotter location with one of those incredible two-story booths at MWC 2016.

Follow Seth Colaner @SethColaner. Follow us @tomshardware, on Facebook and on Google+.