PCI Express 4.0 Brings 16 GT/s And At Least 300 Watts At The Slot (Update: Not Quite)

Update, 8/24/16, 2:06pm PT:PCI-SIG reached out to tell us that the power increase for PCI Express 4.0 will come from secondary connectors and not from the slot directly. They confirmed that we were initially told incorrect information. We have redacted a short passage from our original article that stated what were originally told, which is that the slot would provide at least 300W, and added clarification:

• PCIe 3.0 max power capabilities: 75W from CEM + 225W from supplemental power connectors = 300W total
• PCIe 4.0 max power capabilities: TBD

New value “P” = 75W from CEM + (P-75)W from supplemental power connectors.

To ensure optimum max power, it is recommended that card and system implementers refer to the associated PCI Express specifications that include high-power thermal and mechanical guidelines. Implementers should review also refer to the latest versions of the PCI Express x16 Graphics 150W-ATX Specification (including the supplemental power cable) and the PCI Express 225W/300W High Power Card Electromechanical Specification for additional information.

Original article:

The next iteration of PCI Express will double the bandwidth per lane, but that shouldn't surprise anyone. During Intel Developers Forum, we sat down with Richard Solomon, Vice President of PCI-SIG, and found the hidden gem in the next specification; it provides at least a 4x increase in power delivery at the slot, which might eliminate the need for auxiliary power cables with some GPUs.

The PCI-SIG consists of more than 730 paying members that provide input and a board of directors that help steer the process. The standards body is just "days away" from releasing version 0.7 of the PCIe Express 4.0 specification to members, according to Solomon. The final specification should come by the end of 2016, but the group will have an additional 0.9 step in between before completion at 1.0. Some companies, such as Mellanox, are shipping leading-edge PCIe products with the 0.7 spec, but will update when version 1.0 arrives. The group is already working on PCIe 5.0.

As we sat through our presentation with Salomon, I kept thinking back to the AMD Naples server motherboard the company displayed the previous night. AMD didn't want to divulge too many details about the next generation server platform, but a picture is worth a thousand words. If you look closely, there is a single 20-pin power connector like any other modern motherboard. There are also four 8-pin and two 6-pin power connectors. AMD marked the PCIe power connectors as follows:

P0 ABCD PWR
P1 ABCD PWR
P0 EFGH PWR
P1 EFGH PWR

These connectors are in addition to the processor 4+4 pin connector marked P0 and P1 CORE PWR. Even with a 64-core platform, that much power for the board and processors would severely limit AMD's chances of getting into data centers. We feel that AMD added the extra power for something else, like the PCIe edge connectors.

When we asked the PCI-SIG, we received the news that for the first time, PCIe will get a massive power increase at the connector. Solomon couldn't recall the exact ceiling because member companies have proposed several options. (We were originally told that it would be at least 300W, with a ceiling of 400-500W, but PCI-SIG has since disconfirmed that by stating that some of the power will come from secondary connectors.)

PCI Express 3.0 provides a maximum of 75 watts thus far, though many slots support the default 25W. The rating for a x16 lane card comes from 3 amps at 3.3 volts and 5.5 amps at 12 volts, which is a rating that dates back to the introduction of PCI Express 1.0a in 2003.

Speeds and Feeds

The next iteration of the PCI Express standard will officially pass the 1-gigabyte bandwidth mark and knock on the door of 2 gigabytes per second, per lane, per direction. The performance shifts from 3.0 to 4.0 while x16 devices like video cards get a massive headroom increase that will aid in providing the bandwidth needed for next generation virtual and augmented reality. PCI Express is a full-duplex protocol that can send and transfer data at the same time. The speeds listed above are for single direction performance. PCIe 4.0 uses the same encoding scheme as PCIe 3.0.

PCI Express 4.0 Usage

Image 1 of 5

PCI Express has moved well beyond video cards. Currently, nearly every IO device routes through PCIe to send signals back to the CPU. We expect to see rapid adoption of the new standard for storage and networking products.

NVMe SSDs using PCIe 3.0 x4 recently reached the usable limit of the interface, such as the Samsung SM961 that reads data at 3,100 MB/s. Enterprise NVMe SSDs using the add-in card form factor and bifurcation will ship later this year with PCIe 3.0 x16. The increase in throughput will allow manufacturers to reduce costs by delivering the same performance with half the number of PCIe lanes. Analyst firm Forward Insights predicts enterprise PCIe storage devices will surpass the number of SATA products shipped between 2017 and 2018. 

At Computex 2016, Marvell told us to expect consumer networking products with 10-gigabit speeds in January (CES time frame). 10GbE will finally fall out of the enterprise sector and into the laps of us at home and provide roughly 1GB/s of data transfer speeds over commodity CAT 6 cabling. A single 10GbE connection will only require a single PCIe lane with the 4.0 specification.

Mobile, Internet of Things (IoT), and other battery-operated devices will benefit from low-power states and a new focus on burst performance. PCIe 4.0 adapts new L1 sub-states with half- and quarter-swing bursts, which use just 400 and 200-millivolt steps.

Connecting External Devices Through A Standardized PCIe Cable

Image 1 of 2

During the last year, we've seen a number of companies build external graphics housings that route back to a small form factor PC via external cabling. The big fear is that a product from Company A will not work with a product from Company B. The PCI-SIG looks to end interoperability issues with the OCuLink standard. 

Storage systems will also adopt the PCIe-over-cable OCuLink connection. The cable is more graceful than SATA Express and the reworked SAS cables that we currently use for U.2 devices.