Skip to main content

AMD's EPYC CPUs Push Netflix Server Bandwidth To 400 Gbps

AMD EPYC CPU
AMD EPYC CPU (Image credit: AMD)

Netflix has been serving up to 200 Gbps of TLS-encrypted video from a single server since 2020. Nonetheless, the company aims to double the bandwidth to 400 Gbps. During his presentation at the EuroBSD 2021 conference (via HardwareLuxx), Andrew Gallatin, Senior Software Engineer at Netflix, detailed the challenges of pushing the bandwidth envelope on its FreeBSD-based servers.

Netflix turned to AMD's EPYC Rome processors to achieve its goal. The company equipped its server with the EPYC 7502P, which wields 32 Zen 2 cores with a 2.5 GHz base clock and 3.35 GHz boost clock. More importantly, the 32-core beast offers up to 128 PCIe 4.0 lanes, good for about 250 GBps of bandwidth or around 2 Tbps in networking units. Netflix paired the EPYC 7502P with 256GB of DDR4-3200 memory, with a total memory bandwidth of up to 150 GBps, or 1.2 Tbps in networking units.

For storage, Netflix's AMD-powered server utilizes 18 Western Digital WD SN720 2TB NVMe SSDs. It's also equipped with a pair of Nvidia's Mellanox ConnectX-6 Dx network adapters that communicate through a PCIe 4.0 x16 interface. Initially, Netflix was only getting 240 Gbps out of the server, primarily due to the limitation on the memory.

Netflix experimented with different NUMA (Non Uniform Memory Architecture) configurations to maximize the bandwidth. AMD's EPYC processors support different NUMA nodes per socket, which can either be 1, 2 or 4. Naturally, the processor dictates which modes are available or not. The EPYC 7502P, which is the SKU used in Netflix's server, supports all three NUMA modes. According to Gallatin's slide, a single NUMA node configuration delivers up to 240 Gbps, while a setup with four NUMA nodes bumps the value up to 280 Gbps.

In an attempt to optimize the performance and avoid hardware bottlenecks, Netflix tested offloading the TLS encryption to the Mellanox ConnectX-6 Dx, instead of the EPYC 7502P. With a bit of tinkering with the software and some firmware updates, Netflix managed to squeeze 190 Gbps per Mellanox ConnectX-6 Dx adapter or 380 Gbps with two network adapters. The encryption no longer passes through the processor, so it helps free up resources and cuts memory bandwidth by half. The results showed 50% processor utilization, with four NUMA nodes and around 60% without NUMA.

Image 1 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 2 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 3 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 4 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 5 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 6 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 7 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 8 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 9 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 10 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 11 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 12 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 13 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 14 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 15 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 16 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 17 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 18 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 19 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 20 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 21 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 22 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 23 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 24 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 25 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 26 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 27 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 28 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 29 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 30 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 31 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 32 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 33 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 34 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 35 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 36 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 37 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 38 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 39 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 40 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 41 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 42 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 43 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 44 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 45 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 46 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 47 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 48 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 49 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 50 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 51 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 52 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 53 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 54 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 55 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 56 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 57 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 58 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 59 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 60 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 61 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 62 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 63 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 64 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 65 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 66 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 67 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 68 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 69 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 70 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 71 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 72 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 73 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 74 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 75 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 76 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 77 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 78 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 79 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 80 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 81 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 82 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 83 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 84 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 85 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 86 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 87 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 88 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 89 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 90 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 91 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 92 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 93 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 94 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 95 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 96 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)
Image 97 of 97

Serving Netflix Video at 400 Gb/s on FreeBSD

Serving Netflix Video at 400 Gb/s on FreeBSD (Image credit: Netflix)

Netflix Server Configurations

AMDIntelAmpere
ProcessorEPYC 7502P (Rome)Xeon Platinum 8352V (Ice Lake)Altra Q80-30
Memory256GB DDR4-3200256GB DDR4-3200 @ DDR4-2933256GB DDR4-3200
Storage18 x WD SN72020 x Kioxia 4TB NVMe (PCIe 4.0)16 x WD SN720
Network Adapters2 x Mellanox ConnectX-6 Dx2 x Mellanox ConnectX-6 Dx2 x Mellanox ConnectX-6 Dx

Netflix evaluated other processor options from Intel and Ampere, but AMD was clearly the superior option. For example, the EPYC 7502P offered 280 Gbps, while the Xeon Platinum 8352V (Ice Lake) and Altra Q80-30 delivered 230 Gbps and 180 Gbps, respectively. 

The memory was the bottlenecked on the Intel system, since the Xeon Platinum 8352V natively supports DDR4-2933 as opposed to the EPYC 7502P's DDR4-3200 support. Gallatin expects similar performance with the EPYC 7502P if the Ice Lake chip is paired with equivalent memory. While the Altra Q80-30 from Ampere does support DDR4-3200 memory, the chip is limited to 180 Gbps.

Nevertheless, the Altra Q80-30 was the closest competitor to the EPYC 7502P with the TLS offload. The system offered 240 Gbps, but Gallatin noted low processor utilization and many output drops, which could be a PCIe-specific problem. After enabling extended tags, the Altra Q80-30 system pumped out 320 Gbps, just 60 Gbps lower than the EPYC 7502P system. Apparently, the Xeon Platinum 8352V system had the PCIe relaxed ordering option locked out, so Gallatin wasn't able to assess the performance of the network adapter.

While 400 Gbps sounds good and all, Netflix already has a 800 Gbps prototype for testing. Gallatin didn't share the specifications, but he hinted that we may hear about it next year.

  • watzupken
    AMD denied Intel another big client again.
    Reply
  • Rdslw
    looking at what are the options, that prototype might be special dual core board OR ddr5 with next gen single core. Both exciting options for us.

    I really cant wait for ddr5 and all the goodies they bring.
    Reply
  • lazyabum
    Good for everybody except consumers.
    Reply
  • King_V
    lazyabum said:
    Good for everybody except consumers.
    Huh?
    Reply