Power Consumption Results
Intel has this cadence where its latest architectures roll out on the mobile/desktop side, and are then followed in the high-end workstation/server space. We saw the benefits of 22 nm manufacturing first from Ivy Bridge, and then with Ivy Bridge-EP. Then, Haswell integrated the platform's voltage regulation circuitry for tighter control of power on-package. Needless to say, we were expecting a notable improvement from Haswell-EP to follow some of the gains already measured using desktop-class offerings, and we got it.
Haswell-EP incorporates a number of technologies that we anticipated bettering the power consumption story. First, it brings much of the power delivery from Intel's first- and second-gen Xeon E5s on-package. Haswell-EP can control P-states on a per-core basis, allowing them to be granularly spun up and down as demand dictates. Intel claims up to 36%-lower power consumption from its Per-Core P-States (PCPS.)
Then there's DDR4 memory, which, in addition to increasing data rates, also employs a lower input voltage. In most desktops, sub-1 W savings per module doesn't mean much. In a server, however, where you might have eight DIMMs per CPU and multiple processors per node, hacking away at total platform power a few watts at a time adds up quickly.
For companies looking to upgrade from three-generation-old Westmere-EP processors, there is another major difference. The new Wellsburg PCH (Intel C610 series) runs extremely cool and can control 10 SATA-based drives using standard ports. Back when Westmere was modern, the only way to get lots of PCIe connectivity in a server for expanded storage was adding a second IOH36 chip. That component required a decent sink and plenty of airflow to cope with its heat. When two of them were in a system, they became a significant cooling consideration. Since Haswell-EP employs 40 lanes of PCIe 3.0 on-die, and is mated to a C610 PCH manufactured on a newer process, you get big platform power savings compared to pre-Sandy Bridge-EP systems. Servers sporting Westemere-EP started coming off of lease en masse in early 2014, so they're the ones most likely to be replaced by Haswell-EP.
To generate some hard data, we took our 1U Supermicro test bed and allowed it to idle with no PCIe expansion cards installed (only the on-board networking controllers were active). The results were awesome:
Remember, our Haswell-EP-based server sports two 135 W Xeon E5-2690 v3s and 16 eight-gigabyte memory modules. It also uses redundant cooling, which is great in a datacenter environment, but not particularly power-friendly. Even still, the takeaway is that the Haswell-EP-based system's idle power consumption is extremely low.
Then we fired up three instances of well-threaded tests using c-ray 1.1, sysbench CPU (prime solver), and STREAM concurrently. The results were interesting; mainly, the Xeon E5-2699 v3 drew quite a bit more power than we were expecting. Granted, in many of our performance benchmarks, those same CPUs deliver greater than 2x improvements over the first-gen Xeon E5-2690. That's what will trigger consolidation of older machines into fewer Xeon E5-2600 v3-based boxes during this refresh cycle.