Introduction
When Apple shipped the iPhone 5s with a custom designed 64-bit CPU, it took the mobile industry by surprise. The move to 64-bit was inevitable, but nobody expected Apple to get there so quickly, including Qualcomm whose 64-bit CPU was just a dot on a long-term roadmap. Without a custom designed core of its own, Qualcomm adopted ARM’s Cortex-A53 and Cortex-A57 stock cores for its flagship Snapdragon 810 processor last year.
Working from a less than ideal position resulted in a less than ideal SoC. Even before the Snapdragon 810 made a public appearance, there were rumors about overheating and memory controller issues. Our own testing validated the overheating rumors—a product of pairing the power-hungry A57 core with TSMC's 20nm HKMG process—and we have yet to see an 810 use the full bandwidth available from LPDDR4-1600 memory, even the v2.1 revision.
Even though the 810 was a stopgap, it was not all bad. The Adreno 430 GPU improved upon the Adreno 420 in Snapdragon 805, maintaining Qualcomm’s lead in ALU performance, and a faster Category 9 X10 LTE modem moved next door to the CPU after getting booted off the SoC island in the 805.
Still, it’s hard to feel anything but disappointment when it comes to the 810. Excessive thermal throttling held back performance, forcing the A57 CPU cores to sit idle. In some scenarios, the older Snapdragon 801 and 805 SoCs, along with some mid-range all A53 designs, offered equivalent or better performance. An unenviable position for a flagship product.
Qualcomm hopes to move past these issues with the Snapdragon 820 and Kryo, its first custom-designed 64-bit CPU. Qualcomm’s goal for the 820 is not just about improving performance, however. It's also about enabling innovative user experiences by leveraging heterogeneous compute, which combines the unique abilities of each processor—CPU, GPU, DSP, and ISP—to maximize performance and minimize power use. Computer vision, advanced imaging, and virtual reality are all targeted applications.
Zeroth
Many of these new capabilities will be made possible by Zeroth, a machine learning and computer vision API that developers can use to take advantage of Snapdragon 820’s hardware. This “cognitive computing platform,” as Qualcomm calls it, should further improve the capabilities of virtual assistants on smartphones and also anything that requires more human-like intelligence. One of the ways it does this is by mimicking how humans learn through positive reinforcement. We’ve already started to see mobile devices incorporate intelligent behaviors, but these generally leverage the processing power of cloud computing. With the 820, Qualcomm believes this processing can now be done locally on the device, improving privacy as a consequence, since all of that unique user data will not have to be processed on someone else’s servers.
Qualcomm’s Scene Detect technology is the application of Zeroth to computer vision. Again taking advantage of heterogeneous computing, it uses neural networks for scene detection, object recognition, and pattern matching for both still images and video captured by the device’s camera. There are many uses for this technology, including the automatic tagging of photos for easier search retrieval and augmented reality. The video above demonstrates the basic capabilities of this system.
Smart Protect will be one of the very first “killer applications” of Zeroth. A technology that goes beyond traditional signature based antivirus protection, it will be able to identify “abnormal behavior,” such as noticing that the phone is taking pictures when the screen is locked or sending SMS messages without user interaction, by using machine learning and behavioral analysis. This feature can be used to identify either zero-day malware or “transformational malware,” which is malware created to bypass popular antivirus software.
This feature has a component running at a low-level within the Android kernel and another part running within Qualcomm's SecureMSM secure execution environment, which should make it much harder for malware to circumvent. This also puts Smart Protect in a position to effectively monitor system resources, app communication, etc.
Heterogeneous Computing Examples
Beyond Zeroth, Snapdragon 820 uses heterogeneous computing to enable a host of advanced imaging features. One example takes advantage of OpenCL 1.2 and FastCV APIs to post-process a video stream in real time, separating and blurring the background to increase privacy during a video conference. By combining the processing power of both the CPU and GPU, Qualcomm claims performance improves by more than 2x compared to using just the CPU alone and also reduces power consumption by up to 40%. This same technology is also used to improve the quality of panoramic images, removing stitching seams and cleaning up ghosting artifacts caused by moving objects. Further applications could include providing a real-time preview of video effects while recording or improving augmented reality experiences.
Qualcomm’s improveTouch feature, which is also present in the Snapdragon 810 SoC, moves functionality from an external touchscreen controller onboard the SoC. Utilizing the DSP and low-power CPU island, this improves touch latency and enables more sophisticated noise rejection algorithms. The improved processing enables sophisticated water droplet rejection, essentially making the screen usable when wet, and improves touch sensitivity while charging the device by filtering out EMI. There’s also an ultra-low power double-tap screen wakeup feature.
Tying all of the specialized processors together in an efficient manner is the job of Qualcomm’s Symphony System Manager. According to Qualcomm, “Symphony is designed to manage the entire system-on-chip in different configurations so that the most efficient and effective combination of processors and specialized cores are chosen to get the job done as quickly as possible, with the least amount of power”. That’s no easy task, so we’re eager to see what real-world battery life is like when products start shipping.
Now that we understand Qualcomm’s vision for Snapdragon 820 and its future SoCs (it’s heterogeneous computing if you have not figured it out) and some of the experiences it enables, it’s time to take a closer look at the hardware.
MORE: Best SmartphonesMORE: All Smartphone ContentMORE: All Tablet Content
