AMD Epyc Surpasses Dual-Socket Intel Xeon in Netflix Server Optimization
Netflix boosted its AMD Epyc-based single-socket server's encrypted video data bandwidth to about 200 GBps.
Netflix's data center engineering team took on a mission recently to double the encrypted video bandwidth of the video streaming company from 100 Gbps to about 200 Gbps. The company reached its goal on both Intel and AMD-powered servers, with the AMD Epyc servers seeing triple the bandwidth compared to before.
Intel Xeon Server Improvements
As detailed by Netflix senior software engineer Drew Gallatin at the EuroBSDcon 2019 conference and detailed by Phoronix, Netflix was serving 100 Gbps of encrypted video data from a single server since 2016 using Intel Broadwell Xeon CPUs to power its FreeBSD servers. The Broadwell CPUs feature a memory bandwidth of 60 GBps and 40 PCIe 3.0 lanes supporting 32 GBps of I/O bandwidth.
The company’s newer server processors include Intel Skylake and Cascade Lake with 90 GBps memory bandwidth and 48 PCIe 3.0 lanes with support for 38 GBps I/O bandwidth. But in order to get to its encrypted video bandwidth goal of 200 Gbps, Netflix needed at least 64 PCIe 3.0 lanes.
How could Netflix achieve that? The answer was to “throw another CPU socket” at the problem, according to Gallatin's presentation, doubling the CPU sockets per server.
As a result, Netflix's Intel server went from 105 Gbps to 191 Gbps. The new dual-socket Intel-based server has a 180 GBps memory bandwidth, 96 PCIe 3.0 lanes and about 75 GBps of /IO bandwidth.
AMD Epyc Server Improvements
Netflix also uses AMD's "Rome” Epyc server chips for encrypted video data. With AMD’s chips, there wasn’t a need to do dual-socket, as Netflix could use a “quad chiplet” Epyc CPU per server. The AMD CPUs support 120-150 GBps memory bandwidth, 128 PCIe 3.0 / PCIe 4.0 lanes with 100 GBps I/O bandwidth (200 GBps for PCIe 4.0).
AMD CPUs started out significantly slower than the Intel counterparts, serving only 68 Gbps of encrypted video data per server. This was likely due to FreeBSD not having as many optimizations in place for the new Zen architecture and Epyc server chip platform.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Once the Netflix team applied all the optimizations they could, the single-socket Epyc server was able to catch-up and even slightly surpass the dual-socket Xeon server, tripling its initial performance from 68 Gbps to 194 Gbps (compared to an ultimate 191 Gbps for Xeon).
The Netflix team did criticize AMD for not yet having well-developed tools for server operating systems, such as Linux. However, with the rise in popularity of AMD’s Epyc server chip platform, we can expect AMD to invest much more aggressively in the development of its tools.
Lucian Armasu is a Contributing Writer for Tom's Hardware US. He covers software news and the issues surrounding privacy and security.
-
jimmysmitty ReplyThe Netflix team did criticize AMD for not yet having well-developed tools for server operating systems, such as Linux.
Thats one area AMD has a ton of work to do and will need to invest a lot of time and money. Its one thing Intel is extremely far ahead of AMD.
Its why I say the server market will be a very hard won market for AMD compared to desktop. More cores is not the only thing to consider there. -
JamesSneed Replyjimmysmitty said:Thats one area AMD has a ton of work to do and will need to invest a lot of time and money. Its one thing Intel is extremely far ahead of AMD.
Its why I say the server market will be a very hard won market for AMD compared to desktop. More cores is not the only thing to consider there.
This is likely AMD's largest hurdle. They need money to hire coders so hopefully enough early adopt EPYC. This is also where I worry if Intel gets serious they will have a leg up on GPU's because if they want they can put more resources on drivers than even Nvidia. -
jimmysmitty ReplyJamesSneed said:This is likely AMD's largest hurdle. They need money to hire coders so hopefully enough early adopt EPYC. This is also where I worry if Intel gets serious they will have a leg up on GPU's because if they want they can put more resources on drivers than even Nvidia.
Intel already has more resources dedicated to it. People don't realize how big Intels software side is. Hell they don't realize how involved in everything computer wise Intel is. Its why I laugh when people say they will never support Intel again, because no matter what you buy Intel has had a hand in some part of a modern PC.
That is also why I don't subscribe to AMD just absorbing massive amounts of server market share. There is so much more to it than "moar cores for less!!!" but people want to ignore a lot of it.
In the end AMD will take some market share from Intel in the HPC/server space. But until they can provide the same level of service on every front it will be minimal and I am sure Intel will do their best to see to that, after all HPC/Servers is very high margin.