Intel Plays Defense: Inside Its EPYC Slide Deck
Memory & Bandwidth Projections
- amd8
- amd9
- amd10
EPYC has eight memory channels, outnumbering Purley's six, and supports up to 2TB of memory per processor. If you go the Intel route, you'll need a more expensive "M" SKU to accommodate up to 1.5TB. Most mainstream Xeons only support up to 768GB of memory.
Intel claims that EPYC's memory access will suffer due to AMD's multi-die implementation. Each CCX (Core Complex) comes armed with two channels of DDR4. Access to the rest of the channels incurs a latency penalty. According to Intel, that reduces per-core local memory bandwidth to 42.6 GB/s, which is in-line with AMD's per-channel 21.3 GB/s bandwidth specification (170 GB/s per socket).
AMD offsets those penalties by establishing four different NUMA (Non-Uniform Memory Access) nodes per EPYC processor, or eight nodes per dual-socket server. During its workshop, Intel said that'll restrict the platform's scaling in many workloads. In contrast, Intel's mesh architecture provides direct access to all six memory channels, which is purportedly better suited to in-memory databases and HPC applications.
- amd11
- amd13
- amd14
- amd15
- amd16
- 01
Intel doesn't have access to EPYC processors (though we're guessing it'll be one of AMD's first customers). Instead, the company used a 95W eight-core Ryzen 7 1800X to project AMD's base frequency for the 180W parts. Basic math dictates that AMD has to drop the per-die TDP to 45W to fit four of them into a 180W envelope. This led Intel to guess at a 2.2 GHz base frequency. AMD announced its product line-up after the workshop we attended, and Intel's frequency projections were close.
Since it didn't take a stab at AMD's boost clock rates, Intel locked its own Skylake cores to a similar 2.2 GHz and conducted a series of memory/cache latency and memory/PCIe bandwidth tests. The company claims EPYC's access to "far" L3 cache will weigh in at roughly the same latency as going to DDR4, which our sister site AnandTech confirmed in its testing. Intel also pointed out that EPYC's "near" L3 access is actually faster than Skylake's. That might benefit some workloads.
Each Zeppelin die includes 32 PCIe 3.0 lanes. According to Intel, EPYC's per-core PCIe bandwidth outweighs what the memory controller can do, creating an imbalance when copying data into memory. Intel also believes the 50 GB/s interconnect bandwidth (which AMD officially specifies at 42.6 GB/s) creates contention and restricts EPYC's usable PCIe and memory bandwidth across different dies and sockets.
Simultaneous multi-threading testing was achieved by toggling the feature on Ryzen 7 1800X. Apparently, Intel found that it didn't deliver as much of a boost as Hyper-Threading. The server-side Java test even slowed down, which Intel conceded could be due to a bug in the software or BIOS. AnandTech recorded an SMT scaling advantage for the EPYC 7601 during the SPEC CPU2006 SMT integer performance test. The processor offered an average boost of 28% compared to Intel's 20%.
- amd17
- amd20
- amd21
- amd22
Gartner predicted in 2016 that many organizations had virtualized over 75% of their infrastructure, so VM performance is a key requirement for modern deployments. Intel claimed that EPYC has "No VM Interoperability" with VMware, KVM, Xen, or Hyper-V. But then, after Intel's presentation, both VMware and Microsoft made an appearance at AMD's launch event. They are listed, along with Xen and KVM, on AMD's hypervisor partner list.
Intel also pointed out there isn't a way to support live VM migration between Intel and AMD systems. Of course, the general idea is that it will be difficult to migrate from Intel's platforms to AMD's EPYC. It also creates an isolated VM compute silo if you use AMD processors. The work-around would be simply powering down the VM and migrating between platforms.
Doubling down on its opinion of EPYC for virtualized workloads, Intel argued that VMs spanning multiple CCXes, such as a six-core VM, would suffer inconsistent performance due to higher L3 latency. Moreover, moving out to 10 cores would span two dies, imposing even worse performance due to die-to-die interconnect latency. These theories do make technical sense. But without actual silicon, it's impossible to quantify the impact for certain.
Notice that the "Desktop Die" messaging pops up in these slides as well...
MORE: Best CPUs
MORE: Intel & AMD Processor Hierarchy
MORE: All CPUs Content
Phhh, your platform is getting trash in floating point calculation... 50%. And thanks for the thermal paste on your high end chips... no thermal problems involved.
To be fair, many of those "cheap shots" were fired before AMD announced or clarified the features Intel pointed fingers at.
That said, the number of features EPYC mysteriously gained over Ryzen and ThreadRipper show how much extra stuff got packed into the Zeppelin die. That explains why the CCXs only account for ~2/3 of the die size.
It's taking us back to the old A64.
- They quote WTFBBQTech.
- Use the word "desktop die" all over the place without batting an eye on their own "extreme" platform being handicapped Xeons.
- No word on security features. I guess omission is also a "pass" in this case.
This reads more like a scare threat to all their customers out there instead of trying to sell a product. Miss Lisa Su is doing a good job it seems.
Cheers!
The extra server-centric stuff (crypto superviser, the ability for PCIe lane to also handle SATA and die-to-die interconnect, the 16 extra PCIe lanes per die, etc.) in Zeppelin didn't magically appear when AMD put EPYC together... so technically, Ryzen chips are crippled EPYC/ThreadRipper dies.
I don't know if you're agreeing or not... LOL.
Cheers!
Just pointing out that the "cut down chip" argument works both ways.
Imagine for a moment: Intel attempting to pitch an unpredictable TCO to a datacenter that encounters a worst-case situation only when using one of the most attractive features of the product. That won't end well for Intel, and I suspect they know it. Especially in IaaS applications.
It's also worth mentioning that there's no standard for determining TDPs, and varies from manufacturer to manufacturer.
I would point you to Anandtech's power related portion of their review, I would place even odds on Intel not bringing up TDP being the real world results on those numbers. Those numbers didn't look that rosy for Intel, to me, in Anandtech's very rapidly written and published early comparison review.
It would appear to me, that Intel's PR department is not very confident in their product from presentation of many of their slides.
Can not wait to get my mitts on these new chips (of both flavors) myself.
With some space between the dies as in the Epyc CPU - you've spread out the thermal load (not a lot, but somewhat) which combined with the soldered IHS should help the Epyc to run cooler.
Meanwhile, you have 18 or so concentrated cores in Intels monolithic design and while that might help with intercore latency, it doesn't do much good if the CPU has to run slower because of thermal limits.
Also, according to the Anandtech tests, the AMD Epyc 7601x2 vs Intel Xeon 8176x2, Idle power usage was 151w for the Epyc system, vs 190w for the Xeon system. Under MySQL loads, 321w for Epyc, 300w for Xeon. Under POVRay - 327 for Epyc vs 453 (!!!) for Xeon.
All in all, being +/- 20w isn't too bad, but that 120w margin for Xeon in the POVRay testing was rather surprising.
On the L3 near vs far metric. Its only fair to mention the fact that there are near and far L3 caches on the xenon as well. So it will also be a concern of intel chips. Its going to take 9 link hops to get from the core in the upper left to get down to the lower right. Thats going to add a lot of latency.
The other concern with the mesh network. Is routing between cores. Im just going to assume intel has done their homework here and is routing intellegently. But if they havent, there will be bottle necks arround the memory controllers and in the center of the chip.
Again its another consistency issue that will have to be watched.
Neither Intel's nor AMD's design is bad, they both have trade offs. Which is to be expected with this level of scaling.
Not trying to dismiss the issue tho, just saying its an issue on both platforms.
You gotta tune your workload to your hardware.
We mentioned the duplicated vendors at the top of the last page.
Equally stupid was the implication that AMD hasn't been talking to these companies. It's by far the worst PR nonsense from Intel I've ever seen. The whole thing reads like some kid in a schoolyard yelling, yeah, well my dad could beat up your dad! Nuuuhhh! Unbelievable. Linus Tech Tips covered it here:
https://youtu.be/f8sXQ6JsNu8?t=20m23s
Ian.
Indeed it does. Thing is, Intel is the one pointing it out here, not AMD. At least, so far, I haven't seen any AMD presentation with this level of snarky remarks.
I mean, I have to admit I did had a chuckle at "glued dies", because, ironically enough, we all called that the Pentium Ds and C2Q's back then. Intel was just relieving all that accumulated anger from those years when AMD had the "real" dual and quad core variants.
Cheers!
The biggest thing for me that affects all of their performance material here and raises my skepticism to even healthier levels, is the Notices and Disclaimers slide:
Yes, Intel's compiler optimisations are old topic of discussion, and now it feels like pointing to a flogged dead horse and saying that we should flog it some more, but it's still an important point to consider.
Funny some of the same ones have been made by AMD. I remember AMD ripping on Intels C2Q for using MCM designs.
You realize this is in the data center, a market that the price of just the CPU is not as big as to an individual, correct? Also it has nothing to do with the desktop market and how they are affected.
Did you not read the article or do you assume all Intel CPUs are for consumers?
Talking or not I understand Intels point there. Not being a very active member of a lot of markets in the server space, not just data center, brand new uArch that is still very young means that the software devs will have a lot of work optimizing for AMD specifically.
It is not a simple matter of "We talked to them and it will work". Hell VMWare has a ton of issues even on Intel platforms. Sometimes a single VMWare driver will just kill VMs due to bugs.
It is a lot of work and AMD needs to be prepared to work very closely with those companies and that will also cost quite a bit to maintain if they expect to truly be competitive.
To be fair Intels compiler will optimize for every possible Intel supported extension. And the thing is that AMD does not support all of the same ones. Hell AMD only started SSE 4.1/4.2 support with Barcelona while Intel does not support SSE 4a at all. Intel has AVX-512 while AMD is still running AVX-256.
I would not expect Intels compiler to optimize as well for AMD as others and in this market where Intel easily dominated for years I am not surprised that their compiler is the most widely used.
I personally think AMD should build their own compiler as they know their uArch best.
Reduce your power by 50%, you also reduce A/C costs by a substantial amount.
The name of the game, all things more or less equal == cost of operation.