Apple Scores AR Eye-Tracking Coup With SMI Acquisition

It’s no secret that Apple is planning to build an AR headset of some kind (and its ARkit endeavors are already gaining traction), but with the purchase of SMI (SensoMotoric Instruments), the company has, in one fell swoop, revealed a key part of its plan and also undercut the plans of some of its XR competitors.

The acquisition of SMI was done quietly, as is typical of Apple. (The same thing happened when Apple snapped up Metaio, an AR company with promising IP, for example.) According to a shareholder resolution Tom’s Hardware acquired (from a source of publicly available documents in Germany, where SMI was headquartered), Vineyard Capital Corporation acquired all the shares of SMI. Vineyard is essentially Apple; the Roll Of Deeds document was signed by Gene D. Levoff, in Cupertino. Mr. Levoff is listed by Bloomberg as the Vice President of Corporate Law and Director, Apple Operations International.

Image 1 of 2

To be clear, this is not a licensing deal of some kind--Apple bought SMI outright. We presume this is a play not just for SMI’s IP, but for its human talent, too. That means SMI’s engineers are likely coming along for the ride and will be moved to Cupertino.

Apple’s Big Get

Eye tracking is one of the holy grails for the next wave of XR devices. It only makes sense; whether in VR or AR, if your HMD knows where your eyes are looking, that has an impact on graphics demands (foveated rendering), interacting with virtual characters, what might pop up on your HUD, and so on. Generally speaking, it can work with the IMU to understand where you are in space, even on an inexpensive headset that can otherwise perform only 3DoF.

SMI has been demonstrating its technology since at least Mobile World Congress in 2016, when the company showed us eye tracking demos in VR using tiny and dirt-cheap hardware. The size and the minimal cost (under $10) are both significant factors in the value of SMI’s eye tracking technology. Because of how minute it is, it can fit into or onto virtually any VR HMD or AR glasses. It could also be added to devices such as iPhones and iPads to provide eye tracking for a number of applications, a la Tobii’s HMD-less approach to gaming.

For Apple, then, the SMI acquisition may offer multi-pronged value. Eye tracking in its phones and tablets is one thing; adding it to the company’s laptops and desktops is another; and the implications for Apple’s AR efforts are yet another entirely.

A Blow To The Industry

Even as Apple delights in its latest acquisition, the implication for the rest of the industry is perhaps a bigger story. It’s not as if SMI was just a couple of guys in a garage hoping to be spotted and swooped up by a big company; it’s been working with a number of other companies, none of which--presumably--will be able to continue using it, unless Apple decides to license the technology for some reason.

Most notably, SMI was working with Valve on integrating eye tracking into a Vive HMD,  and SMI also scored a partnership with Qualcomm, which is itself an emerging key player in the XR market. SMI got to the point with Valve, at least, where it was actually giving journalists hands-on time at GDC this spring.

Now, though, instead of finding its way into headsets from some of the most important HMD makers, SMI’s eye tracking tech will likely be exclusive to Apple.

The Remaining Players

Even though the sudden exit of SMI from the wider eye-tracking market is going to shake things up (and likely not in a good way), it’s not the first, nor will it be the last. Late last year, Oculus snapped up The Eye Tribe, for example. But there remains multiple companies developing eye tracking for XR.

There’s QiVARI, a collaboration between EyeTech Digital Systems and Quantum Interface. We haven’t heard from from this group for a while, but Quantum Interface, at least, has been working on its software.

Fove VR built its entire HMD around eye tracking. We’ve been watching this tech develop since CES 2016; over the course of the year, Fove has outed a more refined design, confirmed system specs, and opened up orders for dev kits. Even though Fove ostensibly is an HMD maker, we imagine that a larger company may want to license or acquire its eye-tracking IP more than anything else. We’d be a bit surprised if Fove eventually pressed on as an HMD maker.

Arguably the most formidable player in the eye tracking game is Tobii, a multi-faceted, international company. Indeed, Tobii’s IP runs wide and deep, with foci in both research and gaming, and both within and without HMDs. The closest we’ve seen Tobii come to infiltrating the high-end VR world, though, is when we saw a working demo of the eye-tracking tech in an HTC Vive at GDC this spring.

Assuming that Apple will find a way to implement SMI’s eye tracking technology on iPhones, iPads, MacBooks, and iMacs, and not just AR glasses, Tobii is the most direct competitor. We’ve now seen Tobii’s technology work on laptops, monitors, smart glasses, and inside a VR HMD. There’s a certain amount of adaptation and refinement that Tobii may need to perform before its tech makes complete sense on a pair of AR glasses, but we find it hard to imagine the company isn’t already actively working on that very thing.

One wildcard is Microsoft. It sounds as if whatever Apple does with AR glasses will be a consumer-focused version of Microsoft’s decidedly business-focused HoloLens. To date, HoloLens does not offer eye tracking. Otherwise, it’s the king of the AR hill--for now. Still, Apple has been buying XR companies for years now, with the lingering promise of AR...something…but there’s been nothing to show for it, at least publicly. The SMI purchase will certainly help push Apple towards the finish line.