Somewhat quietly, Qualcomm is full bore into the VR/AR/MR business. At CES, Qualcomm announced its Snapdragon 835 SoC--billed as dedicated to “immersive experiences”--and the hardware launch foil for the new chip was Osterhout Design Group’s (ODG) R8 and R9 augmented reality glasses.
It was no coincidence that Qualcomm, one of the most dominant smartphone SoC makers, debuted the 835 on a device that is not a smartphone. The company was sending a clear message that it’s not interested only in smartphone-based mobile VR like Daydream and Gear VR, but in dedicated, standalone HMDs (be they VR, AR, or MR).
Hitting 10mn
Qualcomm made headlines late in 2016 when it announced that it achieved 10nm with the 835 and also built its 48-core Centriq 2400 Processor. We know the former uses Samsung’s FinFET process, but we aren’t certain about the latter. At the time it was announced, Qualcomm promised that 10nm would grant the 835 a 27% increase in performance or up to a 40% decrease in power consumption over 14nm (not to mention a 30% increase in area efficiency).
Assuming that Qualcomm’s claims hold true, the 835 represents a noteworthy opportunity for the burgeoning XR headset market. It’s no secret that the vast majority of the industry is driving towards untethered, all-in-one HMDs for all manner of XR. That’s an obvious “must have” on the AR and MR side, but even headsets built primarily for VR are yearning to be free, if you will; look no further than Intel’s Project Alloy; Oculus’ Project Santa Cruz; and a smattering of third-party wireless VR devices, technologies, and accessories from Sixa, DisplayLink, MIT, Intel, TPCAST, and Quark VR.
In all cases, what you’re essentially doing is cramming a PC onto a headset. To say that’s ungainly and difficult would be an understatement. You have to balance a great deal of compute power with energy efficiency and weight, and then you still need to figure out what to do with all that sensor data.
The 835 chip promises to address some of those issues. It’s also important to note that there's no separate visual processing unit (VPU) here to offload some of the work from the CPU and GPU. There isn't even a dedicated inertial measurement unit (IMU). Qualcomm leveraged the existing DSP functionality of the 835 to enable headset makers to skip the IMU altogether. According to Jim Merrick, Qualcomm's IoT Marketing Director, "An IMU is essentially a special purpose DSP. We already have a DSP … that’s integrated into our SoC in the first place."
Keep in mind that the Intel Project Alloy prototype has a Skylake-based Atom chip for now, and even the next generation will have just a Kaby Lake CPU--notably, still not a 10nm Cannnonlake processor. Even Microsoft’s HoloLens, which is an exceptional device, relies on a combination of an Intel Cherry Trail chip and a dedicated “HPU” (Holographic Processing Unit). According to Qualcomm, HMDs built with the 835 will enjoy a physically smaller chip with better efficiency without sacrificing any meaningful performance.
Further on the road to efficiency, Merrick told us that all of Qualcomm’s VR reference designs have eye tracking built in, because eye tracking is a “prerequisite” for foveated rendering. Foveated rendering is an answer to the problem of having to render a whole virtual environment, which is a waste of resources if you consider that the human eye can only see so much of a scene at any given time and with any reasonable amount of focus.
When you think of a mobile HMD and Qualcomm’s value, you may think of a lightweight AR headset like ODG’s R8 or R9, but the company is making plays on more heavier-duty, VR-oriented headsets, too. Merrick acknowledged that Qualcomm has been talking with the PC makers who are promising to deliver VR HMDs this year:
A lot of the PC makers are also handset manufacturers, so we have existing relationships with these people. [But] we’re getting a lot of interest. Some of our solutions like the 6DoF, or the single the monocular camera--those reduce cost and shorten time to market. It’s very interesting to people. Our GPU is very robust, we’ve seen a lot of interest in it. Our thermals are very good. I think everyone agrees you want to get to an untethered, all in one type device.
Supporting Operating Systems
Hardware aside, one of the key advantages that Qualcomm offers concerns supported operating systems--or, perhaps more accurately, supported content platforms.
Merrick was adamant about the need for content:
The content pipeline is critical. You know, Oculus has captured the imagination of the industry, specifically, and consumers to a somewhat lesser extent, for the last couple of years. But what has saddled Oculus is a content licensing and delivery platform. There’s some fantastic demos, some great technology there, but there’s not a robust ecosystem. That’s why Sony had the opportunity with PlayStation VR to get better traction. It’s helped Vive with Steam, because they have a content delivery and licensing platform. It’s all about the content.
When it comes to XR, Qualcomm has powerful, superb platforms to offer: Android and Windows. Technically, as Merrick was quick to point out, Qualcomm has always tried to be OS-agnostic. But one aspect sorely lacking on ARM chips (on which Qualcomm’s Snapdragon SoCs are based) was always Windows support. (Steve Ballmer can tell you all about that.)
With an announcement back in December, though, that’s suddenly changing: x86 emulation is coming to ARM, and guess which company will be the first to support it? (Qualcomm. It’s Qualcomm, if you missed the hint.)
If Qualcomm enables Windows to run on its chips, that means the Windows Holographic shell should work, as well. Windows Holographic has its own set of mixed reality applications, of course, but according to Microsoft, it will also support any UWP apps. Therefore, a headset built on the 835 could feasibly run any modern Windows application and Windows HoloGraphic apps.
This is in addition to whatever is in the Google Play Store, because the 835 is already set up to run Android. That includes Daydream apps and, depending on the hardware, Project Tango as well.
And of course, any device should be able to leverage whatever emerges from the WebVR effort being spearheaded by Facebook/Oculus (with their Carmel browser), Google, and Mozilla.
Looking Into The Future
The 835 looks to be a platform upon which Qualcomm rides into the future. It’s built to support deep learning and eye tracking, for example--two technologies that may become rather standard on many XR devices at some point.
Eye tracking will certainly become a go-to feature for many XR HMDs (possibly a must-have at some point). It has great potential as an input method, and as we mentioned, you need it for foveated rendering. SMI is the darling of Qualcomm in this regard; Merrick called the company “the leaders in the space.”
Another extremely compelling technology we’ll be seeing in HMDs is a deep learning component, and on that point, Qualcomm believes the 835 is ready for it. Said Merrick, “Snapdragon 835 has substantial compute capability to run one or many deep neural networks across all cores (CPU, GPU, DSP).” Developers also get to use the Snapdragon Neural Processing Engine SDK.
“This software framework gives developers and OEMs the tools to run trained neural networks on the Snapdragon core of choice based on the power and performance demands of the chosen user experience or feature. Supporting these workloads on the device ensures optimal performance as opposed to relying on a connection to the cloud,” added Merrick.
| Header Cell - Column 0 | Snapdragon 835 SoC |
|---|---|
| CPU | -Qualcomm Kryo 280 CPU, 8 cores, up to 2.45GHz, 64-bit-10nm |
| GPU | Adreno 540 (supports OpenGL ES 3.2, OpenCL 2.0 full, Vulkan, DX12) |
| DSP | -Qualcomm Hexagon 682 DSP-Qualcomm All-Ways Aware technology |
| Display Support | -Max on-device and external display: 4K Ultra HD, “up to 4K”-Up to 60fps-Up to 10-bit color depth, Rec2020 color gamut, Ultra HD Premium ready |
| Memory | 1,866MHz dual-channel LPDDR4x |
| Storage Support | UFS2.1 Gear3 2L, SD 3.0 (UHS-I) |
| Camera Support | -Qualcomm Spectra 180 image sensor processor, 2x ISP, 14-bit, Qualcomm Clear Sight camera features-Up to 16MP dual-camera, up to 32MP single-camera-Hybrid Autofocus, Optical Zoom, hardware-accelerated face detection, HDR video recording |
| Video Support | -Capture: Up to 4K UHD @ 30fps-Playback: Up to 4K UHD @ 60fps-Supports H.264 (AVC), H.265 (HEVC), VP9 |
| Audio Support | Qualcomm aptX codec technology, Aqstic audio technology |
| Connectivity Support | -Qualcomm Snapdragon X16 LTE modem (1Gbps peak download, 150Mbps peak upload)-Wi-Fi: 802.11a/b/g/n, 802.11ad, 802.11ac Wave 2; 2.4GHz, 5GHz, 60GHz-Bluetooth 5.0-NFC-USB 3.1 |
ODG: The Rider
Given the above, we might say that Qualcomm is a dark horse in the XR field, and if so, ODG is its rider. It’s true that others (see SMI, above) are hitching their wagons to Qualcomm’s platform, but ODG’s R8 and R9 AR glasses feel like a big win for both companies. The buzz around the spectacles at CES was palpable, most notably within ODG’s crowded, multi-story booth, wherein all the ODG reps were sweaty and breathless and excited.
The possibilities that the R8 and R9 promise is enormous, although it would seem that much of that promise is yet to be unlocked. To be frank, the demos ODG gave us at CES left something to be desired. This was partially due to the fact that a few pairs of the glasses that ODG was using for demos were having issues, and there just wasn’t much compelling content to pick from just yet, but the actual demos weren’t especially mind-blowing. Much of it consisted of looking around within 360° videos, so we didn’t even get to experience 6DoF tracking, for example.
However, a rundown of the specifications and features shows tremendous potential. Note that the R8 and R9 are similar in many ways--indeed, it’s difficult to even tell them apart at a glance--but there are key differences. Let’s start with a chart:
| Osterhout Design Group AR Glasses | R8 | R9 |
|---|---|---|
| SoC | Qualcomm Snapdragon 835 (8-core, 2.45GHz) | |
| Display | Dual 720p stereoscopic, up to 60fps | Dual 1080p stereoscopic, up to 60fps (up to 120fps at lower res) |
| Field of View | 40 degrees | 50 degrees |
| RAM | 4GB Pop LP-DDR4 | 6GB Pop LP-DDR3 |
| Storage | 64GB | 128GB |
| Camera | Dual 1080p cameras | Single 14MP 4K (60fps) camera |
| Degrees of Freedom | 6DoF | 6DoF (with MIPI expansion port) |
| Battery | 1,300mAh Li-ion (2x 650mAh | |
| Connectivity | Bluetooth 5.0802.11ac Wi-FiGNSS (GPS/GLONASS) | |
| Weight | 4oz | 6oz |
| OS | Reticle OS on Android 7 (Nougat) | |
| Availability | Fall 2017 | Spring 2017 |
| Price | Under $1,000 | $1,800 |
Both the R8 and R9 run on Android 7 (Nougat) with the Reticle OS layered on top, and they support the Unity and Unreal engines. Further, an ODG SDK is in the works that would let you enjoy content from essentially any Android platform.
Both pairs of glasses run on Qualcomm’s 835 SoC (2.45GHz) and offer 6DoF tracking, dual stereoscopic displays, 1,300mAh Li-ion battery capacity (mounted in the earpiece), and at least one front-facing camera. Note well that compared to the previous-gen R7 glasses, which sport a weaker 805 chip, narrower FoV, only a 4MP autofocus camera, and only 3DoF tracking, the R8 and R9 offer much more impressive features.
Neither the R8 nor the R9 offer much in the way of field of view, which is an unfortunate limitation in terms of immersion. They can muster just 40° and 50° FoV, respectively. It’s worth noting, though, that the R9 can display in 22x9 (native digital movie format) or 16x9 aspect ratios.
Although the R8 is the “little sibling,” a double-take worthy feature it offers over the R9 is dual 1080p cameras--as opposed to the R9’s single 4K camera--which means it can shoot binocular video.
However, ODG built the R9 with an expansion port on top, so you can add all manner of modules to expand its capabilities. (Who exactly is making these modules remains to be seen.) The idea here is that ODG wants to sell consumers a nice all-in-one package with the R8, but enthusiasts and pros can opt for the R9 and add those modules they need for efficient extensibility. That top-mounted port offers MIPI, USB, and power to flow through it. An ODG rep rattled off many potential add-ons for the port, including night vision goggles, medical detectors, LIDAR (Light Detection and Ranging), structured light sensors, or multiple cameras.
Unlike the R8, the R9 also offers numerous onboard buttons, mounted on the ear pieces, including the standard Android buttons. However, that’s just one means of input; an ODG rep told us that really, you could employ almost any conceivable input. That could include voice commands, a wireless MEMS controller, controls via smartphone tethered by Bluetooth to the device, and more.
Both sets of glasses are capable of object tracking, so you can place virtual objects in physical spaces. ODG employs SLAM, fused with IMU data and the camera input.
For now, we’ll give the tepid demos a pass; the devices in the booth had been abused for days, and given that neither the R8 nor R9 is actually on the market yet--they’re coming in the fall and spring of this year, respectively--we expect that ODG is still doing some tinkering.
Expect more XR devices running on the 835 to emerge soon, and expect those devices to land on a range of HMD types from AR to VR. What will be most interesting to watch is whether or not some of the major HMD players gravitate towards this ostensibly smaller, lighter, and more efficient platform over the Intel-based chip solutions in Project Alloy, HoloLens, and other upcoming HMDs.
The race for all-in-one XR HMD supremacy is on, and that’s good for both the market in general and consumers specifically.
