Nexus 6 Performance With Android 5.1

Does the Android 5.1 update bring performance and battery life improvements to the Nexus 6?

Introduction

In our Nexus 6 review, we saw its Snapdragon 805 SoC deliver strong CPU and GPU performance while running the initial Android 5.0 release. Internal storage performance however, was a point of contention.

With Android Lollipop, Google announced that full disk encryption (FDE), an optional feature available since Android Honeycomb, would be enabled by default. This requirement was later revoked due to performance issues on certain classes of hardware, the Nexus 6, having shipped with FDE enabled, being a prime example.

At the conclusion of our initial performance analysis, Google released the Android 5.1 update, which includes some patches meant to address the performance impact of using FDE on the Nexus 6, in addition to changes that affect overall system performance and battery life.

In light of these changes, it's worth taking a second look at the Nexus 6, comparing performance after installing the Android 5.1 update to the initial 5.0 release.

MORE: All Smartphone Articles

CPU And System Performance

Google has made a couple of changes in the 5.1 update specific to the Nexus 6 that will impact system performance. For starters, Qualcomm’s thread migration boost feature has been disabled. Prior to the update, the task scheduler would notify the Qualcomm driver whenever a thread migrated from one CPU core to another, and in an effort to increase system responsiveness, the destination core’s frequency would be boosted to the lesser of the origin core’s frequency or 1.7GHz. The boosted core would remain at the higher frequency for at least 20ms. Given the frequency with which threads migrate across cores—sometimes dozens of times per second—this feature has a nontrivial influence on performance and battery life. The similar but different feature that boosts CPU frequency briefly when a touch event is detected is still active, however.

Some of the performance loss (and battery life gain) from disabling the thread migration boost may be offset by the second big change. The Nexus 6 running Android 5.0, like all of the other devices with four or more cores we’ve tested, only kept two cores online, shutting down the other two to reduce static power drain. An updated Nexus 6 will now keep all four cores available, which eliminates the latency involved when activating cores. It should also help the OS spread system load more evenly and possibly improve responsiveness since running processes are less likely to interrupt each other.

System Performance Benchmarks

BenchmarkNexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
AndEBench ProDevice Score82878160-1.54%
CoreMark-HPC (Base)33353282-1.59%
Memory Bandwidth7843 MB/s7809 MB/s-0.44%
Memory Latency4633 KOps/s4915 KOps/s6.09%
Platform
200.9185.9-7.47%
3D28.2 fps28.7 fps1.79%
AnTuTuTotal Score
52094
50550
-2.96%
Basemark OS II FullOverall136813951.94%
System28512614-8.31%
Memory63176421.00%
Graphics246626015.50%
Web790729-7.66%
Geekbench 3 Single-CoreGeekbench Score105110570.57%
Integer106110852.26%
Floating Point8828880.68%
Memory13741342-2.29%
Geekbench 3 Multi-CoreGeekbench Score32503198-1.62%
Integer38763726-3.87%
Floating Point326932970.86%
Memory19641944-0.99%
PCMarkWork Performance42414234-0.17%
Web Browsing40644040-0.58%
Video Playback3566393010.19%
Writing47774243-11.19%
Photo Editing467247972.68%

Looking at the results in the table above, we see a mixture of plusses and minuses. Most of the deltas are small, less than plus or minus 3% and within the tolerance band for these tests; Since many of these synthetic tests keep all available cores fully active, they are not affected by the changes included in the 5.1 update. Where we do see a small loss in performance are less intense CPU-centric workloads such as AndEBench Platform, Basemark OS II System, and PCMark Writing. These tests do not keep all of the cores fully tasked, and subsequently are more sensitive to the thread migration boost. Also, with all four cores active in 5.1—with one or two cores focused on the foreground app and the remaining cores handling storage I/O or processing a background task for example—there’s less thermal/power envelope to set individual cores to their max frequency. This is what happens in the PCMark Writing test, where two or more cores are generally active (there’s several small, interspersed storage reads/writes) but no single core goes above 1497MHz. Running Android 5.0, the Nexus 6 holds two cores at max frequency (2649MHz) for the duration of the test.

The Basemark OS II System test provides another example of the differences in frequency scaling. Both the Math and Multi-Core tests load all available cores at 100%, so we don’t see any difference in behavior or performance between the two different OS versions. Performance in the Single-Core test varies quite a bit when running the 5.1 update, sometimes scoring a little better and other times a little worse than the initial release. It’s the XML Parsing test where we see a significant performance drop-off when running the 5.1 update. With Android 5.0, two cores run at max frequency for the duration the test. After the update, the max frequency is achieved only sporadically, with both cores at max frequency for just a couple short bursts. The end result is less average processing power available in this particular scenario due to the changes made in 5.1.

Web Browsing Benchmarks

BenchmarkNexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
Browsermark12591258-0.08%
JSBench340.2 ms344.0 ms-1.11%
Peacekeeper953908-4.72%
Google Octane43224254-1.58%

The update does not seem to impact JavaScript performance in our browser benchmarks.

MobileXPRT 2013

TestNexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
Overall Performance238209-12.18%
Apply Photo Effects24.62 s29.91 s-21.49%
Create Photo Collages15.11 s18.57 s-22.94%
Create Slideshow26.50 s29.50 s-11.32%
Encrypt Personal Content57.94 s67.55 s-16.59%
Detect Faces6.64 s6.57 s0.99%
Overall User Experience10097-3.00%
List Scroll60 fps60 fps0.00%
Grid Scroll60 fps60 fps0.00%
Gallery Scroll50 fps50 fps0.00%
Browser Scroll59 fps50 fps-15.25%
Zoom and Pinch57 fps57 fps0.00%

MobileXPRT 2013 shows a worst case scenario for the updated Nexus 6. All of the content related tests read and write small amounts of data to disk and don’t fully task the CPU cores. The lower scores here likely reflect the loss of thread migration and a small reduction in storage performance for certain workloads (more on this in the next section).

The user experience tests are unchanged except for Browser Scroll, which sees a significant reduction. Investigating further reveals that the Nexus 6, running either Android 5.0 or 5.1, sets the CPU frequency to a fixed 1497MHz during UI interactions such as the List Scroll, Grid Scroll, Gallery Scroll, and Zoom and Pinch tests. When running 5.0, the Nexus 6 sets the CPU frequency to a minimum of 1497MHz during the Browser Scroll test but frequently spikes to its max frequency of 2649MHz. The 5.1 update changes this behavior, with the CPU governor frequently letting the frequency drop to 300MHz during the Browser Scroll test, which accounts for the lower score.

To see if these results are valid when using a real web browser, I fired up Opera and loaded a simple page without ads (Wikipedia’s home page). I then continuously scrolled up and down while monitoring the CPU usage and frequency. The Nexus 6 running the 5.0 release locks the CPU frequency at 1728MHz with two cores active. Running 5.1, the CPU frequency is locked at 1497MHz, which is the same frequency as the Browser Scroll test and equal to the touch input boost frequency.

For comparison, the Samsung Galaxy Note 4 running Android 5.0.1, which also uses a Snapdragon 805 SoC, behaves quite differently during these tests. The CPU governor allows the frequency to bounce around quite a bit during all of the user experience tests except for Browser Scroll, where the frequency is set to 1190MHz. With a lower average CPU frequency, the Note 4 performs worse than the Nexus 6 in these tests. I also performed the same basic browser scrolling test in Opera on the Note 4, and it also set the CPU frequency to 1190MHz.

These results suggest that UI responsiveness remains essentially unchanged with the OS update, with the Nexus 6 maintaining its lead over the Note 4 in this critical area. However, these benchmarks fail to capture the effect of having all four CPU cores active all the time. In the real-world, there will be several background tasks running, performing tasks like checking for new email, social media news, and app updates. It’s these uncontrolled scenarios where having the additional cores available will help balance the load and help keep these other processes from interrupting UI update threads. So while peak UI responsiveness is unchanged, or even slightly reduced, the effective responsiveness seen in normal use may be better with the 5.1 update.

Storage Performance

The specific issue vexing the latest Nexus is that the hardware encryption engine provided by its Snapdragon 805 SoC is disabled, forcing all file system encryption/decryption to be executed in software on the CPU.

In November and early December of last year, during the development of the 5.1 update, there was a series of software commits enabling and disabling the hardware based encryption engine. Ultimately, hardware encryption was disabled again for the Nexus 6 5.1 update. We asked Google about this decision, but they would not provide an answer.

Even though file encryption is still funneled through the CPU, the update should still improve storage performance. The encryption engine can now utilize NEON instructions and has additional threads at its disposal. Previously, with only two CPU cores online, the encryption driver was allocated only two threads to handle file read and write requests. The 5.1 update however, keeps all four cores active all the time, which means the encryption driver now has up to four threads to perform its work.

AndEBench Pro Storage Test

 512B
SW
512B
RW
4KB
SR
4KB
SW
4KB
RR
4KB
RW
16KB
RR
256KB
SR
256KB
RR
Galaxy Note 4 (5.0.1)449102520877334020425799256653125891127268
Nexus 6 (5.0)2097239084132274495951129692310724857
Nexus 6 (5.1)169636871498261165676150603093232646
Nexus 6
% Diff
-18.9%-12.0%-4.1%-25.7%-17.9%-4.6%16.1%33.9%31.3%
First letter: S=sequential, R=random
Second letter: R=read, W=write
Values in KB/s - Higher is better
File Size: 5, #Folders: 3, #Files/Folder: 1

The results from the AndEBench Pro storage test show nice gains reading larger blocks of data, just over 30%. Unfortunately, reading and writing smaller blocks of data generally show a performance regression. Increasing the file size parameter does improve the numbers, causing a few negatives to flip positive, but the overall trend remains. Based on these results, it appears that disabling Qualcomm’s thread migration boost feature has a greater impact on file encryption performance than adding more threads or using NEON instructions.

What this test fails to capture, however, is the storage performance in real-world scenarios. Reading and writing to storage does not happen in isolation. Instead, the encryption driver must fight threads from the active program, background tasks, operating system, user interface, etc. for CPU time. Having all four cores active all the time means more resources are available to the system and eliminates the latency involved in bringing the other cores online. In theory at least, storage performance in these noisy scenarios should at least be more consistent with the 5.1 update.

Even with the uplift in some scenarios, the Nexus 6 running Android 5.1 still trails the Galaxy Note 4 by a significant margin. If FDE is going to be a viable option, which it needs to be, then Google needs to resolve the issues surrounding full hardware encryption.

Battery Life

The changes discussed above will also affect battery life. Disabling the thread migration boost feature should provide a net power savings. Having all four cores active means less CPU pressure, so the governor can keep individual core frequencies lower on average, again reducing dynamic CPU power. Of course keeping those extra two cores online will increase static power draw due to leakage, offsetting some of the battery life gains.

Battery Benchmarks

BenchmarkNexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
GFXBench 3.0Performance19.5 fps21.0 fps7.40%
Lifetime181 min175.5 min-3.04%
PCMarkWork Battery Life329 min354 min7.45%

Battery life is generally difficult to quantify due to the huge variation in potential workloads, and becomes even more difficult when trying to assess the impact of the specific changes included in the 5.1 update. The best test we have for this is PCMark, which performs a few common tasks instead of purely synthetic loops. Here we see more than a 7% gain in battery life, which equates to a noticeable 25 minutes of runtime.

GPU And Gaming Performance

The 5.1 update brings support for OpenGL ES 3.1 to the Nexus 6, which was missing from the initial 5.0 release. Beyond this, I could not find any other changes that should affect GPU performance.

GPU Benchmarks

BenchmarkNexus 6
Android 5.0
Nexus 6
Android 5.1
Difference
3DMark: Ice Storm UnlimitedScore2361821253-10.01%
Graphics2565122294-13.09%
Physics1848918267-1.20%
Basemark X: Medium QualityPerformance3031526339-13.12%
Dunes: Offscreen32.82 fps28.03 fps-14.61%
Dunes: Onscreen23.65 fps23.03 fps-2.64%
Hangar: Offscreen44.45 fps38.98 fps-12.31%
Hangar: Onscreen34.24 fps31.01 fps-9.43%
Basemark X: High QualityPerformance2082519178-7.91%
Dunes: Offscreen29.04 fps26.84 fps-7.59%
Dunes: Onscreen20.56 fps20.26 fps-1.46%
Hangar: Offscreen25.76 fps23.65 fps-8.17%
Hangar: Onscreen17.38 fps16.93 fps-2.59%
GFXBench 3.0Manhattan Offscreen17.0 fps18.4 fps8.89%
Manhattan Onscreen12.0 fps12.1 fps0.75%
T-Rex Offscreen37.0 fps39.1 fps5.47%
T-Rex Onscreen26.5 fps27.5 fps3.88%
Alpha Blending Offscreen10694 MB/s11344 MB/s6.08%
Alpha Blending Onscreen9281 MB/s9755 MB/s5.10%
ALU Offscreen140.5 fps141.5 fps0.75%
ALU Onscreen59.5 fps59.0 fps-0.06%
Driver Overhead Offscreen25.0 fps24 fps-3.28%
Driver Overhead Onscreen22.0 fps19.5 fps-11.01%
Fill Offscreen7334 MTexels/s7465 MTexels/s1.79%
Fill Onscreen8474 MTexels/s8490 MTexels/s0.19%
Render Quality: Standard2503 mB PSNR2503 mB PSNR0.00%
Render Quality: High Precision3633 mB PSNR3628 mB PSNR-0.14%

The 5.1 update on the Nexus 6 reduces 3DMark and Basemark X performance by up to 15%. Even though these tests primarily stress the GPU, the CPU still plays a supporting role. To understand this better, we logged CPU and GPU frequency while running Basemark X.

As expected, the GPU runs at its max frequency of 600MHz during each test segment regardless of OS version. With the initial Android 5.0 build, the Nexus 6 keeps two cores offline and the other two above the 1497MHz threshold most of the time and at least one core at max frequency fairly often. CPU behavior with the 5.1 update looks very different. No single core runs at max frequency for very long, with cores taking turns jumping from 300MHz to 2649MHz and back down again. There’s also several gaps where all four cores are sitting at 300MHz. Looking at the raw data also shows that only one or two CPU cores are above idle at any given time, so having all four cores online does not provide any performance advantage in this particular benchmark.

Things aren’t all bad, however, as we see small performance improvements in the GFXBench Manhattan, T-Rex, and Alpha Blending tests, indicating that some games will see an uptick in performance.

Conclusion

The Android 5.1 update brings several improvements to the Nexus 6, including a number of bug fixes and UI refinements. It also enables OpenGL ES 3.1 support and the Qualcomm VP8 hardware video decoder. Unfortunately, it does not enable hardware based storage encryption or fix optical image stabilization for the rear camera. There’s also no improvement in display calibration.

The other Nexus 6 specific changes are a mixed bag. Utilizing NEON instructions for the encryption engine does speed up data reads for larger block sizes, but seems to hurt performance when reading and writing smaller data blocks. Disabling Qualcomm’s thread migration boost feature reduces performance by up to 20% in specific workloads where the CPU cores aren’t fully utilized. However, this did contribute to a noticeable gain in the PCMark battery life test.

The final big change in the 5.1 update for the Nexus 6, keeping all four cores online all the time, is harder to evaluate. Typically, our phones perform a number of tasks in the background keeping us updated on events happening in the world. With only two cores generally available, which was the case when running the initial 5.0 build, these tasks could interrupt the foreground app or UI interaction, producing a noticeable lag and hurting the user experience. Having more cores available to service these background tasks should improve responsiveness. The downside is that static power drain is increased, reducing the battery life gained from disabling the thread migration boost, and the thermal/power envelope is decreased, which can keep cores from reaching max frequency and reducing performance.

In the end, the performance penalty seen in specific cases is not enough to outweigh the benefits from the 5.1 update. It’s important to have realistic expectations though. I don’t expect these changes to be permanent, but rather stepping stones on a path to a more efficient CPU governor.

Matt Humrick is a Staff Editor at Tom's Hardware, covering Smartphones and Tablets. Follow him on Twitter.

Follow Tom's Hardware on Twitter, Facebook and Google+.

This thread is closed for comments
10 comments
    Your comment
  • ZolaIII
    Just to point out how CPU clocking logic effect busses on Qualcomm SoCs. For instance if 1 core CPU frequency fell down under the minimum frequently than is tied to max bus frequency it will narrow & memory bandwidth & this will impact GPU performance badly under GPU intensive tasks. For me it looks like that on-demand scheduler is working more as it should under 5.1. Including patch sets from Linux kernel 3.12~3.21 (on demand to use more mid frequencies) should be just enough to address performance regressions still savings (even litle more) juice. They should really disable file encryption on any ARM V7 build. Switching to last stable GCC should increase user experience greatly, it's funny they still use 4.6. To address possible fluctuations & determine real impact of changes do the tests again with performance governor & disable MP decision (hot plug) for GPU tests.
  • zodiacfml
    Ouch. Glad I bought the N5 last year.
  • endeavour37a
    I have an old LG G2 and it does not even have filters :(
  • theusual
    Be sure to upgrade 'Android System WebView' through Google Play as well as for me it didn't upgrade automatically and was causing app issues.
  • Plyro109
    Sorry. Browser seems to be autofilling and submitting in every comment thread I visit.
  • Tracy Kohler
    Quote:
    Get that stupid bar off the top of my screen please! It takes up a whole INCH of screen space (and vertical space is LIMITED already on this wide CRAP they call monitors today). I guess I'll have to come up with a way to kill it and still be able to navigate, other than scrolling. Go back to pages with a bar at top or bottom for index. While your at it FIRE the guy/gal who decided an inch of real estate on a monitor is OK to block all day. He/she will only piss off users over and over (hired from win8 team?...LOL).
  • Senecaz
    TL, DR : Lollipop s*cks! next update please...
  • musical marv
    1987127 said:
    TL, DR : Lollipop s*cks! next update please...
    Why do you say Loliipop sucks?
  • musical marv
    1987127 said:
    TL, DR : Lollipop s*cks! next update please...
    Back Up what you post here and do not ignore it.
  • endeavour37a
    I thought Tom's was a place we could have opinions and points of view we could share and express freely, how does one back up what they like and don't like? I sort of like 5.0 but it's just fine if some else does not, I ignore a lot of stuff myself.