Supermicro is showing off its upcoming machines based on AMD’s 6^th Generation EPYC ‘Venice’ processors, Instinct MI450 accelerators, and, of course, Nvidia’s Vera Rubin-based solutions. Arguably, the most important product that the company demonstrated at its CEO’s keynote is the VR200 NVL72 rack that uses Vera CPUs and Rubin GPUs. The machine, just like other upcoming liquid-cooled systems from Supermicro will use the company’s all-new coolant that has a 1000x higher electrical impedance compared to today’s mixtures, which may be quite important for next-generation AI machines.

“Our new coolant [uses a new] formula [and] reaches up to 1,000 times higher electrical impedance than a standard cooler,” said Charles Liang, chief executive of Supermicro, during his keynote speech at Computex. “In case there are small leaks, when you have a high electrical impedance, the system will not [shut down] [and will] keep running.”

Conventional water-based coolants used in direct liquid cooling systems have electrical conductivity (albeit lower than water), so if coolant leaks onto a motherboard, GPU, power delivery circuitry, or connectors, it can create leakage currents or even short circuits. A coolant with 1000 times higher electrical impedance is far more resistant to current flow, which reduces the likelihood that a minor leak will immediately shut down a system or damage electronic components. This is important as modern rack-scale AI solutions like Nvidia’s VR200 NVL72 are rumored to cost around $8 million, so their protection is crucial. Also, reducing downtime in AI data centers is important as these machines must make money for their owners.

Unfortunately, Supermicro’s claim is difficult to evaluate because the company does not disclose specifications of the coolant, such as conductivity (µS/cm), resistivity (MΩ·cm), or dielectric strength (kV/mm). It also remains tight-lipped about the baseline coolant used for comparison. Since modern water-glycol coolants are already fairly resistant, a 1000-fold improvement sounds significant, but without the actual details, it is impossible to determine the practical magnitude of the advancement. What Supermicro claims is that a minor coolant leak would be less likely to force an immediate server shutdown, which means lower downtime risk in large AI deployments. However, we have no idea whether the new coolant behaves more like a dielectric fluid and less like water-glycol coolants.

The new coolant will be used in all new liquid-cooled systems from Supermicro, including the upcoming AMD Helios and Nvidia VR200 NVL72 machines. Given that Nvidia nowadays does not leave many ways for its partners to differentiate, the coolant with 1000 times higher electrical impedance than conventional coolants will likely be among the key selling points of Supermicro’s Nvidia-based offerings.

Speaking of Nvidia’s AI systems, Supermicro is on track to release its Vera Rubin-based systems in the second half of this year, just like other makers. Supermicro intends to offer both NVL72-rack-scale machines based on Nvidia’s Vera CPUs and Rubin GPUs, as well as Rubin DGX systems with different processors.

As for AMD’s side of matters, Supermicro is also on track to release MI455X-based Helios rack-scale solutions in the second half of the year, though the company does not disclose whether these use UALink or UALink over Ethernet interconnections. In addition, Supermicro is also on track with 1-way and 2-way servers based on AMD’s 6^th Generation EPYC ‘Venice’ processors that are made using TSMC’s N2 process technology. These machines are also expected to hit the market in 2026.

A day after Taiwanese authorities seized 50 Supermicro servers and arrested three people on suspicion of smuggling them into China, Supermicro has clarified that it "worked closely" with law enforcement in Taiwan leading up to the arrests and that it will continue to do so.

This comes days after Nvidia CEO Jensen Huang called on Supermicro to "improve regulation compliance," and months after Supermicro co-founder Yih-Shyan "Wally" Liaw was charged by U.S. prosecutors for allegedly smuggling $2.5 billion worth of servers into China.

Here is the statement in full:

"Supermicro is committed to protecting our advanced technologies and intellectual property, and we are proud to have worked closely with Taiwanese authorities on the recent event, helping to prevent the illicit diversion of our highly sought-after systems into the restricted China market. Our collaboration with authorities in Taiwan resulted in the arrest of three suspects and the seizure of 50 servers that had been deceptively acquired after being sold by Supermicro to an authorized reseller. We thank local law enforcement and legal officials for their vigilance and professionalism.

The initial transaction – Supermicro's sale of products to an authorized reseller – followed a rigorous vetting and review process that exceeded applicable government requirements. This case highlights the challenges that can arise when products are resold through multiple downstream parties beyond direct manufacturer control. It also underscores the importance of continued collaboration across industry and government to strengthen safeguards, enhance supply chain visibility, and facilitate the enforcement of export control laws. Supermicro will continue to cooperate with law enforcement and government officials in the United States, Taiwan and other jurisdictions to ensure our technology is distributed as lawfully intended."

The smuggling operation that led to arrests reportedly routed servers coming from Taiwan through Japan. Supermicro itself claims that the servers were sold to an authorized reseller, presumably in an attempt to wash its hands of any accusations that the company was directly involved.

Although three Supermicro employees have been charged by U.S. officials in connection to an earlier smuggling operation, neither Supermicro nor Nvidia have been charged. Despite this, Supermicro's stock plummeted nearly 30% after news of U.S. charges broke.

The U.S. government, and presumably Taiwanese authorities, have been cracking down on server smuggling to China over the past several months. In March, the Trump Administration proposed a strict licensing system for advanced AI hardware exports, and some U.S. senators have called on a wholesale ban on export licenses for Nvidia hardware to China, as well as several other south-east Asian countries including Malaysia, Thailand, and Vietnam.

Nvidia CEO Jensen Huang called on Super Micro Computer to strengthen its export compliance controls after arriving in Taipei on Saturday, months after U.S. federal prosecutors charged the server maker's co-founder and two others with conspiring to smuggle approximately $2.5 billion worth of Nvidia-equipped servers to China through shell companies in Southeast Asia.

Huang told reporters at Songshan Airport that Nvidia insists its partners follow U.S. trade rules. "We insist our partners are compliant. We hope that they will enhance and improve their regulation compliance and prevent that from happening in the future," Huang said in an address to the media.

Huang's comments came days after Taiwan launched its first formal crackdown on illicit AI hardware exports. The Keelung District Prosecutors' Office announced earlier this week that three suspects had submitted fraudulent shipping declarations to export Super Micro servers containing Nvidia AI chips to China, Hong Kong, and Macau.

The Taiwan case is separate from, but closely related to, the much larger U.S. federal prosecution unsealed in March. That indictment charged Supermicro co-founder Yih-Shyan "Wally" Liaw and two others with conspiring to smuggle approximately $2.5 billion worth of Nvidia-equipped servers to China through shell companies in Southeast Asia. Liaw has pleaded not guilty, and Supermicro has said it’s not named as a defendant and is cooperating with the investigation.

In the same press scrum at Songshan Airport, Huang confirmed that China is included in the $200 billion addressable market he projected for Nvidia's upcoming Vera CPU during the company's earnings call on May 20th. "H200 has been licensed to ship to China. It would be terrific to be able to serve that market. The Chinese market is very important. It's very large, of course," Huang told reporters, according to Reuters.

Despite the licensing approval, not a single H200 has been delivered to a Chinese customer. While roughly 10 Chinese firms have been cleared to purchase the chip, shipments haven’t started, and President Trump's talks with Chinese President Xi Jinping in Beijing earlier this month produced no breakthrough on Nvidia chip sales.

Huang is in Taipei ahead of Nvidia's GTC Taipei event and his Computex keynote on June 1st, where he’s expected to explore the Vera Rubin platform's software stack. He described the platform as "the largest product launch, probably in the history of Taiwan," noting that each Vera Rubin NVL72 system contains nearly 2 million parts and involves around 150 Taiwanese ecosystem partners.

Public documents reveal that four Chinese universities, two of which are known for conducting military research for the People’s Liberation Army, bought AI servers containing export-controlled Nvidia chips from Super Micro Computer. According to Reuters, the Harbin Institute of Technology (HIT), which focuses on missile, satellite, and robotics technologies, bought a system from Super Micro in July 2025 that contained eight Nvidia A100 AI GPUs.

Beihang University, which also conducts aerospace and defense research, is said to have acquired a machine-learning workstation from the company just this March, configured with four Nvidia A100 chips. These two universities are part of the “Seven Sons of National Defense,” a designation for Chinese academic institutions with deep ties to China’s Ministry of Industry and Information Technology and the People’s Republic of China's defense industry.

Nvidia’s A100 chips are less powerful than H200 AI GPUs, which U.S. President Donald Trump has finally allowed for export to China in late 2025. However, potential buyers still need to acquire export licenses from the federal government before they can get their hands on these chips, and it’s unlikely that PLA-linked institutions will ever get approval.

It’s unclear how the universities were able to purchase these Super Micro servers. However, three individuals, including Super Micro co-founder Yi-Shyan “Wally” Liaw, were recently arrested for smuggling $2.5 billion worth of advanced AI servers into China. The conspirators allegedly shipped orders from resellers and shell companies located in different countries in Southeast Asia. Once the servers containing the controlled AI GPUs arrived at the local warehouse, their serial numbers were transferred to dummy servers, and they were then issued new fake documentation. Once completed, they were forwarded to their final destination in China.

News like this has several American lawmakers on edge, with U.S. Senators Jim Banks (R-Ind.) and Elizabeth Warren (D-Mass.) writing a letter to Commerce Secretary Howard Lutnick urging him to suspend the issuance of export licenses to China and its intermediaries until the issue has been resolved. It would be a blow to Nvidia if the administration grants the request, especially as the company has finally received Chinese orders for its H200 chips after several months of uncertainty from Beijing.

Super Micro Computer, Inc. is facing a lawsuit from some shareholders who claim the company failed to disclose its dependence on illegal Chinese sales, resulting in an inflated stock price. According to Reuters, Super Micro is accused of committing securities fraud because its alleged illegal activities made it look like the firm had a stronger business outlook than actual. Furthermore, they said that the company failed to mention that it had issues with export controls compliance.

Three Super Micro employees, including its co-founder, Yih-Shyan “Wally” Liaw, were charged with conspiring to unlawfully divert cutting-edge U.S. artificial intelligence technology to China last week. It’s alleged that these illegal exports of high-end Nvidia chips delivered around $2.5 billion in sales for the company in 2024, which would account for more than 16% of the company’s total $14.94 billion in sales that year. This news led to a massive 33% drop in stock price for Super Micro, wiping out more than $6 billion in value for investors.

According to reports, real Super Micro servers were shipped to warehouses in Southeast Asia, where their serial numbers were removed and switched to dummy servers to fool inspectors. From there, the real AI servers were sent to China through a fake company that had fabricated paperwork. This shows that this was a deliberate, well-thought-out operation, with thousands of the fake servers still sitting across the region, supposedly awaiting delivery to local customers. While the federal government didn’t accuse Super Micro of any wrongdoing, the fact that a big chunk of its sales is tied to alleged illegal activity will hurt its stock price and shake investor confidence in the company.

This isn’t the only case that’s being prosecuted in the U.S. right now that relates to AI chip smuggling into China. Three individuals — one from China and two from the U.S. — have just been indicted over the same issue. The Department of Justice even released images of damning text messages between the conspirators, with one message composing a letter to encourage others to find clients who will “act as pass through (sic) partner for customers in China.” These cases, alongside the dozens of reports of AI chip smuggling from Singapore, Malaysia, and other countries near China, show that there is indeed a “healthy” black market for smuggled Nvidia chips.

President Donald Trump made a complete U-turn in late 2025 after he lifted the ban on H200 exports to China, meaning some of the chips that previously needed to be smuggled out of the U.S. could now be legally sold in the country. In fact, Nvidia CEO Jensen Huang said that the company has already received H200 orders from Chinese customers and that it has received licenses from the U.S. government to deliver them. Still, the uncertainty from Beijing has led some companies to consider purchasing these AI GPUs from the black market, especially as AI hyperscalers across the world race to build the most advanced model possible.

Three Super Micro employees have been charged with conspiring to unlawfully divert cutting-edge U.S. artificial intelligence technology to China. An indictment unsealed on Thursday says Yih-Shyan “Wally” Liaw, Ruei-Tsang “Steven” Chang, and Ting-Wei “Willy” Sun were involved in a smuggling plot that leveraged a middleman Southeast Asian company to fake paperwork and repackage servers that were powered by cutting-edge Nvidia chips. The underhanded operation is thought to have netted roughly $2.5b in sales since 2024.

Super Micro isn’t named in the indictment, but Liaw is a co-founder of the server firm, with nearly half a billion dollars worth of shares under his belt (the shares just dropped 12% in value on this news). Moreover, Chang is a sales manager for Super Micro in Taiwan. Sun is referred to as a third-party broker and fixer who has worked with the other two at Super Micro previously. Liaw (71, a U.S. citizen), and Sun (44, Taiwan) have been arrested, but Chang (53, Taiwan) is currently a fugitive, notes the Justice Department PR.

The transshipment scheme exposed

Basically, billions of dollars of Nvidia-powered servers that shouldn’t have been available to Chinese customers were funneled to the country using a fake front company, which fabricated paperwork and assembled thousands of dummy servers, to fool inspectors. It was far from a casual or opportunistic operation, with a high level of coordinated deception.

Despite the sophistication of the covert documentation, logistics, and supporting operations, some analysts have highlighted the brazen absurdity of the operation. For example, the indicted trio have been accused of maintaining a substantial inventory of thousands of dummy (empty shell) servers in Southeast Asia, supposedly awaiting deployment to local (not China-based) customers.

Analyst Max Weinbach highlights some CCTV image captures showing workers using hair dryers to transfer serial number stickers from genuine servers to the dummy server shells. They would subsequently ship the real GPU-packed servers to China.

Today, due to the ever-shifting sands of U.S. export controls policy, some of the smuggled Super Micro servers might be approved as exportable after following a framework to get a license.

Super Micro and Nvidia aren’t in the FBI’s crosshairs

Super Micro has acknowledged that the accused trio are “associated” with it, but highlights that the company isn’t named as a defendant in the indictment. The official statement says that it has “placed the two employees on administrative leave and terminated its relationship with the contractor, effective immediately.”

At the same time, the server maker distanced itself from the actions of the trio. The smuggling caper was described as “a contravention of the Company's policies,” and Super Micro “maintains a robust compliance program and is committed to full adherence to all applicable U.S. export and re-export control laws and regulations.”

Nvidia sent a similar themed statement to Tom’s Hardware. “Strict compliance is a top priority for Nvidia. We continue to work closely with our customers and the government on compliance programs as export regulations have expanded,” wrote the AI and graphics chip maker. “Unlawful diversion of controlled U.S. computers to China is a losing proposition across the board—Nvidia does not provide any service or support for such systems, and the enforcement mechanisms are rigorous and effective.”

Liaw, Chang, and Sun face some very serious federal felony charges. It is early days in the investigation, but it is possible that these people, in trying to enrich themselves, will end up serving decades (up to 30 years) behind bars, and/or fines, asset forfeiture, and also bans from employment in export-controlled industries.

Supermicro CEO Charles Liang announced on X (formerly Twitter) that he will set up shop near xAI’s Colossus AI supercomputer to help Musk achieve his dream of one million GPUs. Liang said that he plans to establish “local operations/production, validation, service and support.” Although it has been previously reported that Elon Musk shifted $6 billion worth of AI server orders from Supermicro to its competitors because of its financial problems, it seems xAI still plans to order from the beleaguered server supplier.

While Nvidia is likely the biggest winner behind Musk’s planned one million GPUs, which will most probably be powered by a mix of Hopper and Blackwell GPUs, it will also require all the accouterments needed to run a complete supercomputer. This includes servers and liquid-cooling solutions, which Supermicro is known for building. And with such a massive order, it does make sense for Liang and his company to put up local operations to support Musk’s ‘Gigafactory of Compute’.

This move will benefit xAI’s operations, as Supermicro would be able to deliver its products and services to such a massive AI server with much greater efficiency. But aside from that, it would also mean that the company would invest millions, if not billions, of dollars in Memphis and the surrounding area to get its operations up and running in support of the Colossus AI supercomputer.

An investment of this scale would greatly benefit the city and the state of Tennessee, bringing in a lot of money in investments and likely creating several thousand new jobs, both directly and indirectly. However, some of the surrounding communities are also concerned about the negative impact of having such a power-hungry industry setting up shop in their area.

Some experts estimate that the 100,000 H100 GPUs Musk currently has in his Memphis facility need 155MW to run. While it already has approval from the Tennessee Valley Authority (TVA) to receive 150MW, some community members are concerned that this huge demand will cause energy prices to spike for residential consumers and that the quick solution to this problem would be to build a new gas plant.

After all, if Musk brings an additional 900,000 GPUs online in the coming years, then his AI facility would likely require over 1.5GW of power. And if other corporations, like Supermicro, set up shop locally, they would need power, too, thus putting a lot of strain on the local electricity supply grid.

The massive power requirements of AI training have pushed several tech companies to invest in nuclear power. Amazon, Google, Microsoft, and Oracle have already signed contracts to develop small modular reactors or restart old large nuclear plants. Even Meta is getting in on the action, having recently released a request for proposals. However, it seems that Elon hasn’t made any plans to invest in nuclear power just yet.

This development is good news for the labor force in Memphis and the state of Tennessee, especially as it has the potential to bring in new jobs and investments in the state and city. However, it must also consider the impact of such a power-hungry industry entering the area.

xAI, Elon Musk’s AI startup, has shifted all AI server orders from troubled Supermicro to Dell, reports UDN.com. Dell, already among the largest makers of servers, reportedly benefits from this decision, just like its suppliers, Inventec and Wistron. In contrast, losing a multi-billion business to rivals amid a potential NASDAQ delisting could be another devastating blow for Supermicro.

Dell and Supermicro used to supply Musk’s companies, including xAI and Tesla. Musk even appeared publicly with Supermicro’s CEO, Charles Liang, who revealed that xAI had made substantial purchases of Supermicro’s liquid-cooled AI servers. However, after the U.S. Department of Justice began to probe Supermicro for accounting manipulations and alleged export violations to China and Russia and its stock plummeted 35% in one day, UDN says Musk’s companies decided to shift orders away from the troubled company.

Among the largest AI server suppliers, Dell is well-positioned to absorb orders. If true, Wistron, which produces motherboards for Dell’s AI servers and does some assembly tasks, would be one of the biggest beneficiaries of this shift. In fact, Wistron is already expanding its production capacity to meet surging AI demand, particularly in its three Hsinchu facilities in Taiwan and its Mexican operations. Wistron is optimistic about the growing demand for AI servers and aims for triple-digit annual growth.

Inventec, another major supplier for Dell, would also reap the rewards from an order realignment. Inventec has long been involved in AI server production and is one of Dell’s top three global server assembly partners. This year, the company primarily supplied machines based on Nvidia’s Hopper processors. UDN claims the company will be ready to mass-produce Nvidia Blackwell-based machines (powered by B200 and B200A GPUs) in the first quarter of 2025. The company reportedly has spare manufacturing capacity in Mexico, so it will likely be able to produce more AI servers for companies formerly served by Supermicro.

Supermicro’s issues stem from delayed financial filings, putting the company at risk of being delisted from NASDAQ. To avoid delisting, Supermicro needed to submit a plan by November 16 explaining the delay and specifying when the required 10-K annual report would be filed. Since the 16th fell on a Saturday, the company’s final deadline is November 18.

If Supermicro is delisted, it could face serious financial repercussions, including a sharp decline in stock value and the immediate repayment of $1.725 billion in convertible notes, which could severely harm the company.

As GPUs for AI and HPC applications get more power-hungry, they require better cooling. For example, a rack with 72 Nvidia B200 GPUs is projected to consume around 120 kW, which is 10 times higher than a typical rack power. As a result, Blackwell-based servers will mandate liquid cooling. On Monday, Supermicro introduced its datacenter-scale liquid cooling solution to prepare for these next-generation servers.

Supermicro's new complete system includes advanced cooling components, software, and management tools to optimize performance and sustainability. In particular, the solution incorporates Coolant Distribution Units (CDUs), cold plates, Coolant Distribution Manifolds (CDMs), cooling towers, and SuperCloud Composer software for end-to-end management. This setup is intended to lower initial investment costs and the total cost of ownership (TCO) by making datacenters more energy-efficient.

Supermicro's cooling technology is aimed at ultra-dense AI servers, each capable of housing dual top-tier CPUs and up to eight Nvidia HGX GPUs in a 4U configuration as well as up to 96 B200 GPUs per rack. This enables four times the computing density compared to traditional setups. 

One of the key benefits of Supermicro's liquid cooling solution is its rapid deployment capability. Supermicro's modular design reduces the time required to get datacenters up and running, cutting it down from months to weeks, which is crucial for companies looking to upgrade or build new facilities quickly. 

The solution’s liquid cooling capabilities are engineered to handle extreme heat levels, efficiently cooling servers that demand up to 12kW of power and AI racks generating over 100kW of heat, exactly what Blackwell-based NVL72 servers need. 

Supermicro's technology also supports warm water cooling, reaching temperatures up to 113°F (45°C). This feature not only boosts cooling efficiency but also makes it possible to repurpose the heat for applications like district heating or greenhouse warming. 

The SuperCloud Composer software included in the package offers real-time monitoring and control over all components. It helps data center operators manage costs and maintain system reliability, ensuring that operations run smoothly and efficiently. 

Supermicro said that it had recently implemented over 100,000 GPUs using its liquid cooling solution (DLC) in some of the largest AI facilities ever built, along with other cloud service providers (CSPs). 

"Supermicro continues to innovate, delivering full datacenter plug-and-play rack scale liquid cooling solutions," said Charles Liang, CEO and president of Supermicro. "Our complete liquid cooling solutions, including SuperCloud Composer for the entire life-cycle management of all components, are now cooling massive, state-of-the-art AI factories, reducing costs and improving performance. The combination of Supermicro deployment experience and delivering innovative technology is resulting in data center operators coming to Supermicro to meet their technical and financial goals for both the construction of greenfield sites and the modernization of existing data centers. Since Supermicro supplies all the components, the time to deployment and online are measured in weeks, not months."

Fujitsu is collaborating with Supermicro to build liquid-cooled servers by 2027, according to a report by The Register. These liquid cooler servers will be based on Fujitsu's upcoming ARM-based Monaka processor, which is slated to be released in the same timeframe.

The combination of liquid cooling and Fujitsu's energy-efficient Monaka chips is aimed at combating sky-high demand for data center capacity, which has become greater than what can be supplied thanks to various factors, including AI. One of the biggest obstacles preventing accelerated data center capacity development the ability to meet the growing power consumption of modern datacenter chips. By combining the efficiency of the ARM architecture with liquid cooling, Fujitsu and Supermicro hope to offer a market-leading server portfolio for their customers. 

Monaka is the name of Fujitsu's next generation ARM-based datacenter processor. The new chip is aimed at AI, HPC, and datacenter deployments featuring 150 Armv9-A cores with SVE2. Monaka is designed to take full advantage of the power efficiency of the ARM architecture, and Fujitsu has set an ambitious goal of having Monaka be twice as power-efficient as its competitors' chips — not its competitors' current chips, but those that will be made in 2026 and 2027. Monaka will be built on TSMC's 2nm fabrication process.

Fujitsu apparently originally designed these CPUs with air cooling in mind. However, the manufacturer is now shifting gears in this partnership with Supermicro. The main goal is to reduce the size of Monaka-based servers; liquid cooling paired with highly power-efficient processors allows designers to build highly compact cooling solutions. 

It's also likely that liquid cooling Monaka could result in greater power efficiency gains as compared to air cooling. Testing by SMC has revealed that Nvidia's GPU servers are 50% more power efficient when using submersion liquid cooling compared to air cooling. We don't know how exactly these servers will be setup, but Fujitsu and Supermicro have an opportunity to make some of the densest and most power-efficient servers in the world by 2027, if Fujitsu can deliver on its goals for Monaka. 

The U.S. Department of Justice (DoJ) is reportedly investigating Supermicro after Hindenburg Research accused the company of manipulating its financial reports and violations of U.S. export regulations to Russia and China, reports the Wall Street Journal. Supermicro has denied any wrongdoing, but the investigation is in its early stages, and stock prices have dropped in response. The DoJ and Supermicro have not formally confirmed the ongoing probe.

DoJ reportedly probes Supermicro

Hindenburg Research published a report titled Supermicro: Fresh Evidence Of Accounting Manipulation, Sibling Self-Dealing And Sanctions Evasion At This AI High Flyer back in August. The report accused Supermicro of improper accounting practices, related-party transactions, and failure to comply with U.S. export regulations. Supermicro is accused of selling high-tech products to China and Russia. This is after Russia invaded Ukraine in early 2022 and follows strict U.S.-imposed export controls on high-performance processors bound for China. The Hindenburg investigation appears comprehensive — the short seller investor says it lasted three months and included interviews with former Supermicro employees.  

Following Hindenburg's report, Supermicro postponed filing its annual report, stating that it needed to review its internal financial controls, which raised suspicions. Now, the Department of Justice has started its own inquiry into the company, focusing on allegations of financial misconduct. According to the report, a prosecutor contacted individuals with potential knowledge of these practices, including information about a former employee who accused Supermicro of violating accounting rules.

Dealing with Russia

Following the Russian invasion of Ukraine in 2022, the U.S. implemented strict export restrictions on high-performance computers and related technology to Russia. As a result, approximately 46 companies involved in handling these products are now under U.S. sanctions, with two-thirds of these exports consisting of components deemed critical by the U.S. government, potentially intended for military use. 

Despite Supermicro's claims of halting sales and recording no revenue from Russia since the war began, the company's exports to Russia have surged threefold, according to Hindenburg's findings. 

One significant recipient of Supermicro products is Niagara Computers, a supplier linked to a major Russian supercomputer used at a previously secret and now-sanctioned Kurchatov Institute nuclear technology research center. Niagara Computers received $46.3 million in products since the start of the war. Allegedly, these sales were initially facilitated through a California-based distributor but were later funneled through three new Turkish shell companies, one of which was sanctioned for smuggling.  

Additionally, around $30 million in components were allegedly shipped through a Hong Kong-based shell entity to VneshEcoStyle, one of Russia's largest importers of dual-use technology, which is also now under sanctions. This company makes no secret that its business is to 'find the equipment abroad and to deliver it in Russia,' which essentially means evading sanctions.

Trading with China

As the U.S. government has been cracking down on technology sales to China and Supermicro has come under scrutiny, the company's business practices to supply technology to Tianxia purportedly become more sophisticated.  

Hindenburg Research's report alleges that Supermicro exported AI, HPC, and surveillance servers equipped with Nvidia processors to China, even to entities with ties to the Chinese military through resellers. 

The report suggests that Supermicro employed intermediaries (for example, using Taiwan-based Leadtek, which derives about 70% - 80% of its revenue from China) and shell companies to obscure the final destinations of its products, thus evading U.S. export control regulations. It alleges that this practice enabled Supermicro to bypass restrictions and continue business in China despite tightening export laws. 

The report suggests that these actions demonstrate systemic governance failures within Supermicro and a willingness to circumvent trade restrictions to boost revenue and profits. This, of course, raises geopolitical concerns about the role of U.S. companies in advancing the technological capacity of strategic rivals. Hindenburg suggests that Supermicro potentially risked violating U.S. laws designed to prevent sensitive technologies from aiding foreign militaries. However, the report provides no direct evidence of Supermicro selling restricted components (Nvidia's A100 or H100 processors) to China. 

Stock drops 

Supermicro has benefited greatly from the rise of artificial intelligence, with its market value soaring from $4.4 billion in recent years to $67 billion by March 2024. The company is a major supplier of AI servers, which has driven much of its recent success.  

Despite Supermicro's growth, the report has cast doubt on its business practices. Apparently, just selling AI servers to interested parties was not enough. Based on Hindenburg's research, the company sold its servers to entities possibly linked to China and Russia's military amid the war against Ukraine, in which China aids Russia with technology. 

Supermicro's stock dropped by 12% after news of the DOJ investigation became public due to the WSJ report. The AI stock boom, which had significantly boosted the company's valuation, has cooled recently as investors adjust their expectations regarding returns on AI investments. However, when it comes to Supermicro, the company's financial results were allegedly boosted by illegal business practices and supplying U.S. adversaries. 

Sometimes, shortages of tiny components that can hardly be called 'high-tech' cost companies tens or hundreds of millions of dollars. This happened to Supermicro, which could not get enough quick couplings for liquid cooling systems, which made it postpone shipments of products and delayed $800 million of revenue from one quarter to another, reports Economic Daily News.

"Some key new components shortage delayed about $800 million of revenue shipments to July, which lowered our EPS for June and will be recognized in our September quarter," said Charles Liang, chief executive of Supermicro, at the company's earnings call last week (via SeekingAlpha).

Liquid cooling is vital for high-performance AI servers. A liquid-cooling system in AI servers typically comprises six components: cooling distribution units (CDUs), cold plates, cabinets, fan walls, coolant distribution manifolds (CDMs), and quick couplings. Quick couplings connect the coolant flow between the cold plate and the cooling distribution unit (CDU) and are essential for liquid-cooled AI machines.

Some quick couplings are prone to leakage, and their quality is crucial. Due to the high demand for AI servers, which require hundreds of these components per cabinet, the market has also seen a surge in demand for quick couplings. As a result, quick couplings have risen sharply, from $40 to $60 per unit, with buyers willing to pay even more to secure them. This price increase reflects increased demand and major challenges in meeting it due to a shortage of suppliers.

The market for quick couplings is controlled by seven companies, including two based in China. However, due to sanctions related to the U.S.-China tech conflict, Chinese companies face restrictions limiting their ability to supply these components, exacerbating the shortage.

Taiwanese firms such as Global Tek, Fositek, and Lotes are taking advantage of this situation by accelerating their production and certification processes. These companies see a significant opportunity to fill the gap left by the constrained Chinese suppliers and meet the growing demand.

Global Tek is actively working with partners in its Wuxi and Taoyuan plants, with samples currently undergoing testing. The company expects to see revenue from these efforts by the fourth quarter of 2024 or early next year.

Fositek, backed by its parent company Asia Vital Components, has also submitted samples for customer certification and focuses on developing quick couplings. Lotes, meanwhile, anticipates making progress by the end of the third quarter, aiming to capitalize on this high-demand market.

Elon Musk's Texas Tesla Gigafactory is expanding to contain an AI supercomputer cluster, and Supermicro's CEO is a big fan of the cooling solution. Charles Liang, founder and CEO of Supermicro, took to X (formerly Twitter) to celebrate Musk's use of Supermicro's liquid cooling technology for both Tesla's new cluster and xAI's similar supercomputer, which is also on the way.

Pictured together among server racks, Liang and Musk are looking to "lead the liquid cooling technology to large AI data centers." Liang estimates the impact of Musk leading the move to liquid cooling AI data centers "may lead to preserving 20 billion trees for our planet," obviously referring to the improvements that could be had if liquid cooling were adopted at all data centers worldwide.

AI data centers are well known for their massive power draws, and Supermicro hopes to reduce this strain by pushing liquid cooling. The company claims direct liquid cooling may offer up to an 89% reduction in electricity costs of cooling infrastructure compared to air cooling. 

In a previous Tweet, Liang clarified that Supermicro's goal is "to boost DLC [direct liquid cooling] adoption from <1% to 30%+ in a year." Musk is deploying Supermicro's cooling at a major scale for his Tesla Gigafactory supercomputer cluster. The new expansion to the existing Gigafactory will house 50,000 Nvidia GPUs and more Tesla AI hardware to train Tesla's Full Self Driving feature. 

The expansion is turning heads thanks to the supermassive fans under construction to chill the liquid cooling, which Musk also recently highlighted in an X post of his own (expand tweet below). Musk estimates the Gigafactory supercomputer will draw 130 megawatts on deployment, with growth up to 500MW expected after Tesla's proprietary AI hardware is also installed. Musk claims that the facility's construction is nearly complete, and it is planned to be ready for deployment in the next few months. 

Tesla's Gigafactory supercomputer cluster is not to be confused with Elon's other multi-billion dollar supercomputer cluster, the X/xAI supercomputer, which is also currently under construction. That's right: Elon Musk is building not one but two of the world's largest GPU-powered AI supercomputer clusters. The xAI supercomputer is a bit more well-known than Tesla's, with Musk already having ordered 100,000 of Nvidia's H100 GPUs. xAI will use its supercomputer to train GrokAI, X's quirky AI chatbot alternative that is available to X Premium subscribers. 

Also expected to be ready "within a few months," the xAI supercomputer will also be liquid-cooled by Supermicro and already has a planned upgrade path to 300,000 Nvidia B200 GPUs next summer. According to recent reports, getting the xAI cluster online is a slightly greater priority for Musk than Tesla, as Musk reportedly ordered Nvidia to ship thousands of GPUs originally ordered for Tesla to X instead in June. The move was reported to have delayed Tesla's supercomputer cluster's construction by months, but like so much Musk-centric news, exaggeration is highly likely.

Investigations have provided evidence that China-based organizations could still get their hands on sanctioned Nvidia GPUs as recently as Feb 28, 2024. According to Reuters, Chinese universities and research institutes may have sidestepped the sanctions on the most powerful GPUs by buying servers packing these powerful accelerators. However, an Nvidia spokesperson indicated that the products sold were likely to include stocks that had been previously exported to resellers in China.

Tender documents uncovered by Reuters reveal that ten Chinese entities acquired advanced Nvidia chips between Nov 20, 2023, and Feb 28, 2024, by simply ordering servers equipped with them. It is noted that Dell, Gigabyte, and Supermicro server products (including the sanctioned Nvidia GPUs) were sold in China after the Nov 17 expanded embargo, raising questions about sanctions-swerving. Remember, this sanctions policy adjustment even encompassed Nvidia GPUs like the GeForce RTX 4090 consumer graphics card.

Reuters found that the server resellers included 11 little-known Chinese retailers. The sale and purchase of the sanctioned powerful GPUs isn’t illegal in China. Thus, the big unanswered question here is whether the sanctioned GPUs were already in China before November 17, 2023.

In its statement to Reuters, Nvidia told the news organization that the tenders were for products already exported and widely available before Nov 17, 2023. Moreover, Nvidia supported its partners by claiming the documents “do not indicate that any of our partners violated the export control rules.” Nevertheless, Nvidia said it would investigate further.

Of course, it isn’t just Nvidia that will be policing potential sanctions breaches. The U.S. Bureau of Industry and Security monitors the movement of restricted chips and examines cases where sanctions may have been swerved.

Dell, Gigabyte, and Supermicro also responded to inquiries made by Reuters. Dell said it had seen no evidence of servers packing restricted chips being sent to China. It also stated that it would be willing to terminate relationships with resellers found to be breaking regulations and export controls. Gigabyte’s response wasn’t as detailed, with the firm simply stating it complies with Taiwanese laws and international regulations. Supermicro denied any knowledge of third-party exports or re-exports of systems without required licenses. Its U.S. legal firm also asserted that Supermicro goes above and beyond the letter of the law by being proactive about what customers do with their servers.

Earlier this week we saw indications that ample stocks of servers and sanctioned Nvidia H100 GPUs are available to Chinese customers. However, it is difficult to know whether some of these posted offers are clearing existing stocks, genuinely sanctions-busting, or just fraud attempts by scammers.

If you missed the Supermicro Open Storage Summit ‘23, don’t worry. We partnered with Supermicro to create a video recap of everything you need to know. The summit covered topics like next-gen flash storage, AI optimization, and the continual importance of HDDs — if you want to be on the cutting-edge of storage technologies, don’t miss this.

With insights from key players like NVIDIA, AMD, and Intel^®, this gathering of experts is a goldmine for anyone invested in the future of storage technologies. NVIDIA focused on showing how GPUDirect Storage is revolutionizing AI performance, while AMD and Supermicro are meeting the future demands of AI with high-throughput, low-latency solutions. Intel^® also teamed up with Supermicro, exploring the breakthroughs of virtualized storage with accelerated vSAN solutions.

Head to our on-demand sessions to get the latest on key storage trends!

REGISTER NOW and watch on demand. 

The explosion of AI is further heightening demand for storage performance and capacity as organizations feed models and databases with unprecedented amounts of data, meaning the next generation of storage technologies will need to deliver even greater performance, density and capacity than ever before.

Supermicro's fourth annual Open Storage Summit brings together leading storage experts from across the industry including drive manufacturers, compute components manufacturers, software developers and of course Supermicro's industry leading system architects to discuss the latest in storage technologies and how they will solve tomorrow's data challenges from the data center right out to the intelligent edge.

This year’s Summit includes a roundtable keynote session followed by five focus sessions, with guests from the storage industry's leading players including Intel®, AMD, NVIDIA, Micron, Kioxia, Solidigm, and Samsung, as well as Supermicro’s storage software partners. 

New Innovations For Storage Performance

In a time in which pure processing power is game-changing, it’s important to continually reflect on current solutions and look for new ways to keep business players progressing through new levels.  Sometimes, that progress means stopping investment in an old version of that game and crafting a whole new open world instead.

In Session 2 of our 2023 Open Storage Summit, you will hear from NVIDIA on how they are helping organizations build whole new worlds in which to operate. Through the introduction of the third pillar of computing - the Data Processing Unit - DPUs join CPUs and GPUs to create a futuristic blue sky environment in which applications are accelerated well beyond the capabilities of CPUs alone.

This is particularly important in the frenetically growing AI market, in which lightning-fast storage processing time means that critical business initiatives make their way to the leaderboard instead of being relegated to game-over status. 

During this session, players in the audience will: 

• Discover the limitations inherent in traditional storage architectures
• Understand the advantages of GPUDirect storage and RDMA for AI
• Learn how the GPUDirect Storage and RDMA work at the rack-level to combine the resources of multiple systems into one massive compute cluster
• Uplevel their knowledge around how DPUs can effectively offload compute tasks to massively improve storage performance

Watch all of the webinars on demand

Register now for full access to the storage industry's leading online event to get the latest on key storage trends as well as exclusive look into the future of high performance storage from the most influential minds in the industry.

 Join the discussion here! 

REGISTER NOW and watch on demand. 

The explosion of AI is further heightening demand for storage performance and capacity as organizations feed models and databases with unprecedented amounts of data, meaning the next generation of storage technologies will need to deliver even greater performance, density and capacity than ever before.

Supermicro's fourth annual Open Storage Summit brings together leading storage experts from across the industry including drive manufacturers, compute components manufacturers, software developers and of course Supermicro's industry leading system architects to discuss the latest in storage technologies and how they will solve tomorrow's data challenges from the data center right out to the intelligent edge.

This year’s Summit includes a roundtable keynote session followed by five focus sessions, with guests from the storage industry's leading players including Intel®, AMD, NVIDIA, Micron, Kioxia, Solidigm, and Samsung, as well as Supermicro’s storage software partners. 

New Innovations For Storage Performance

Organizations remain increasingly hungry for ever-more dense data storage systems that satisfy their burgeoning data requirements, but they also need to ensure that such systems remain physically and environmentally stable. Ongoing advancements in flash are providing businesses with new menu items that can help them solve critical business challenges while reducing the total cost of ownership (TCO).

During this kickoff session for the 2023 Open Storage Summit, you will discover how technical advances in flash storage, including the introduction of the E3.S form factor and Compute Express Link (CXL), bring a new evolution to what is already a revolutionary technology. In addition, you'll also learn how Supermicro is incorporating these new technologies into the latest generation of systems as well as some of the challenges encountered along the way. 

During this segment, you will:

• Be introduced to new features and improvements in E3.S and learn how they can help transform your business
• Get a peek at the future capacity roadmap for flash, including new form factors such as E3.S, the path to even higher drive capacities, and what’s in store for subsequent generations of storage
• Discover how storage innovations increase performance and lower TCO
• Learn about how CXL can breathe new performance life into critical business applications

Watch all of the webinars on demand

Register now for full access to the storage industry's leading online event to get the latest on key storage trends as well as exclusive look into the future of high performance storage from the most influential minds in the industry.

Register here! 

As workloads evolve and become more data intensive, storage architects can no longer design for worst-case scenarios or take a “one-size-fits-all” approach to meeting their storage needs. Tahmid Rahman, Data Center Director at Solidigm, and Patrick Chiu, Senior Director of Product Management at Supermicro, joined a Tech Talk roundtable discussion hosted by Gestalt IT, to break down the benefits of next-generation quad-level cell (QLC) solid-state storage drives (SSDs) for the enterprise and the cloud.

“Data growth has been phenomenal over the last few years,” said Rahman, noting that this year, “we will be growing 3x the data that we used to have in 2019. And by the year 2025, the data growth will be up to like 181 zettabytes.” The growth is being sparked by user trends, including artificial intelligence and machine learning, which is spurring the need for storage to be modernized, according to Rahman.

Doubling down on density

QLC’s pricing, speed, and capacities compare favorably to even the best hard drive (HDDs) and triple-level cell (TLC) SSDs, making the pure business case for them  a compelling one. However, storage architects should also be aware of another key benefit of QLCs: storage density.  Density allows for more storage in a smaller footprint, which means less physical space is needed and overall energy costs are reduced.

Rahman and Chiu discussed the benefits of high-density QLC storage with Supermicro’s E3 Petascale high performance servers, noting that both Solidigm and Supermicro are doubling down on density. Rahman noted Solidigm’s solutions are “the sweet spot of density and performance combination.” Solidigm is “basically maximizing the PCIe Gen 4 bandwidth on these devices on workload that matters. And at the same time, meeting that insatiable need of density that we’re hearing from most of our Tier 1 customers.”

Making the right size choice

One of the top questions Rahman and Chiu addressed is how a storage architect can choose the right SSD for their workload. The needed read/write ratio can help determine storage needs – many of the applications are tasked with storing and moving massive amounts of data at high throughput, while others are tasked simply with efficiently storing massive amounts of data that are accessed infrequently, Rahman notes. By assessing which category in which their data storage needs fall into enables storage architects to choose the “right-sized” solution, which may come with benefits such as cost savings and sustainability.

Solidigm, realizing greater benefits for density and efficiency, will be offering an E3.S form factor for the first time soon. This new form factor at this capacity will enable up to a petabyte of storage in 1U within Supermicro’s E3 Petascale platform. This offering enables a variety of cloud and enterprise workloads to gain more insights to data, faster, and for much less. Chiu noted that storage architects must keep their total cost of ownership in check. “We are partnering with Solidigm to incorporate their latest storage media to serve the next generation of data center workflow.”

Rahman concluded, “I think we’re marrying the performance and the density together to bring the right SSD to the market.”

Watch the full round table discussion in this Tech Talk featuring Rahman and Chiu. To learn more about QLC and how it is shaping modern workloads, visit solidigm.com/qlc.

About Solidigm: Headquartered in California, Solidigm is a leading global provider of innovative NAND flash memory solutions. Stemming from the sale of the Intel NAND and SSD business, Solidigm became a standalone U.S. subsidiary of semiconductor leader SK hynix in December 2021. Solidigm is powered by the inventiveness of more than 2,000 employees in 13 locations worldwide. For more information, please visit solidigm.com and follow us on Twitter and on LinkedIn.

Although we have been talking about Intel's Ponte Vecchio for over three years now, including pictures from the lab, few of us got to see Intel's Data Center GPU Max 1550 (a product based on the PVC design) in the flesh. That's because the chips were delayed repeatedly and not offered in server systems, but now that's changed, with Intel promising these 8-OAM systems will be available for its customers in July. At Computex, Supermicro decided to correct this wrong and demonstrated its X13 8U 8-way Ponte Vecchio GPU system for AI and HPC as well as the compute GPU itself.

Intel's Ponte Vecchio represents the pinnacle of processor complexity, boasting a transistor count of over 100 billion (excluding memory) and a combined die size of 2,330 mm². The PVC design is a complex arrangement of 47 tiles that includes compute tiles, Rambo cache tiles, Xe Link tiles, and HBM2E memory stacks interconnected using Intel's advanced packaging technologies like Embedded Multi-Die Interconnect Bridge (EMIB) and Foveros. Tiles for PVC are made using a variety of process technologies and then assembled together.  

Supermicro's X13 8U 8-way Ponte Vecchio GPU server itself, it is based on two 4th Generation Xeon Scalable 'Sapphire Rapids' processors rated for up to 350W that are mated with 32 DIMM slots as well as up to 20 hot-swap 2.5-inch SSDs (12 NVMe, 8 SATA).

But while Intel's Ponte Vecchio is extremely complex inside, it looks pretty humble in its OAM form-factor with a heatspreader on. Given the fact that the 100-billion-transistors beast is rated for a 600W TDP, it has an intricated voltage regulating module that includes two high-performance converters each of which is serving its own 'half' of MOSFETs for added granularity of power delivery. By contrast, AMD's Instinct MI250X uses only one high-performance converter onboard, though it is 'only' rated for a 550W TDP. Meanwhile, an OAM slot can supply up to 700W of power, so Intel's module was most likely architected with headroom in mind. 

600W is an exceptional amount of power, so Supermicro's SYS-821PV-TNR uses huge radiators and high-pressure fans to remove the heat from eight of Intel's Data Center GPU Max 1550 processors. These radiators have seven copper heat pipes and dozens of aluminum fins, so even a PVC radiator is a quite an intricate arrangement. 

When we first saw mention of the Supermicro X13SWA-TF motherboard based on Intel’s W790 chipset set to power upcoming workstations as well as high-end desktops based on Sapphire Rapids CPUs, we barely had any details about it. Now, thanks to our colleagues from Chiphell who revealed its picture, we can tell more.

Aas mentioned, the Supermicro X13SWA-TF motherboard is based on W790, a workstation-grade chipset. So it comes in an E-ATX form factor, which is used for workstations, desktops, and tower servers. Speaking of workstations and tower servers, the platform indeed has an Aspeed remote management processor.

One of the first things that strikes the eye is an eight-phase solid-state voltage regulating module for Intel’s LGA4677 processor. To feed the beast, there are two EPS connectors. While the number of phases may not exactly impress overclockers, we are talking about an ultra-high-end product aimed mostly at workstation users who utmost value stability and longevity. For the same reason, the board features a four-phase VRM for its eight-channel memory subsystem.

Being a motherboard aimed at demanding users, the Supermicro X13SWA-TF also features four M.2-2280 slots for SSDs fed directly from the CPU. The board also brings six PCIe x16 slots, five of which are connected to the CPU and therefore support the PCIe 5.0 protocol. Meanwhile, there is no word about exact bifurcation or CXL support, though. Also, there are two U.2 connectors and a multitude of SATA connectors for bulk storage.

Since the Supermicro X13SWA-TF is aimed at a variety of applications, including desktop workstations, it also features an audio codec and appropriate 3.5-mm connectors. 

We know from an early leak that the X13SWA-TF is going to be priced in the ballpark of ultra-high-end enthusiast-grade motherboards, yet the mystery remains when exactly this one will hit the market. For those who have been waiting — considering that now both AMD and Intel have given up on so-called extreme workstations — availability might be the most important question.

A Canadian retailer has listed Supermicro's yet-to-be-announced X13SWA-TF motherboard based on the Intel W790 chipset. It is designed for workstations featuring Intel's Xeon processors in LGA4677 packaging, which points to Sapphire Rapids CPUs.

The Supermicro X13SWA-TF motherboard (listed at Atic.ca and spotted by @momomo_us) comes in an E-ATX form factor, which is used for workstations, desktops, and tower servers. The motherboard is priced at CAD$1290  ($965) with a discount if you pay cash, but there is no mention of its availability timeframe.

Earlier this year, Intel confirmed that its W790 is a workstation-grade chipset (not an entry-level workstation-grade chipset) but never disclosed which processors it supports. But since the motherboard has an LGA4677 socket, it should support Intel's Xeon processors codenamed Sapphire Rapids. 

We can speculate that we might be talking about a special version of Sapphire Rapids CPUs tailored for workstations (i.e., featuring fewer than 60 cores but with higher clocks) or Intel's 34-core Raptor Lake-S processors spotted by our own Paul Alcorn earlier this year. Meanwhile, both Sapphire Rapids and Raptor Lake-S feature high-performance cores based on the Golden Cove microarchitecture and have several things in common. Therefore, it is plausible to assume that Intel's 34-core Raptor Lake processors (also known as RPLS-34C) will be marketed under the Xeon brand, though Intel has never discussed it publicly. 

The Supermicro X13SWA-TF platform has six PCIe 4.0 x16 slots, two 10GbE ports controlled by Intel's X550 chip, and Intelligent Platform Management Interface (IPMI) for remote management, which is handy both for workstations and servers. The basic description does not mention PCIe 5.0 support or M.2 slots (indispensable for desktop PCs), but since this is an early representation of the X13SWA-TF's capabilities, this is not surprising. 

Rumors about a Sapphire Rapids-based platform for high-end desktops and workstations have been floating around for years. However, Intel has never publicly confirmed plans to address the client PC market with its SPR processors as it is focused on launching Sapphire Rapids for servers first. The listing is essentially the first confirmation that Intel's W790 platform supports the company's processors in LGA4677 packaging, which points to Intel Xeon 'Sapphire Rapids' CPUs. Meanwhile, Intel's 34-core Raptor Lake-S processors are perhaps better candidates for mainstream workstations and high-end desktops.

The push for enhanced compute density continues, and servers like Supermicro's 1024US-TRT, which hails from the company's 'A+ Ultra' family, are designed to answer that call with generous compute capabilities paired with copious connectivity options. Supermicro designed this slim 1U dual-socket server for high-density environments in enterprise applications, high-end cloud computing, virtualization, and technical computing workloads. The platform supports up to 8TB of DDR4 memory spread across 32 DIMMs along with plenty of PCIe 4.0 connectivity, dual 10 GbE LAN ports, and up to four NVMe devices in the front bays.

The platform supports dual AMD EPYC 7003 and 7002 processors, meaning it supports up to 64 cores and 128 threads with the EPYC Milan, Milan-X, or EPYC Rome processors. Supermicro's Intel Ice Lake X12 servers can't match that number of cores and threads in a single platform, signifying that the 1024US-TRT offers the utmost density in its portfolio. Naturally, Supermicro competes with other OEM server vendors, like Lenovo, Dell/EMC, and HPE, in the high-volume general-purpose 1U realm with the 1024US-TRT. 

AMD's EPYC Genoa will launch later this year to compete with Intel's incessantly-delayed Sapphire Rapids, setting the stage for either AMD's continued dominance or an Intel resurgence. As we wait for those launches, here's a look at some of our benchmarks and the current state of the data center CPU performance hierarchy in several hotly-contested price ranges. 

Supermicro 1024US-TRT Server

The Supermicro 1024US-TRT server comes in the 1U form factor, enabling incredible density. The server supports AMD's EPYC 7002 and 7003 processors that top out at 64 cores apiece, translating to 128 cores and 256 threads spread across the dual sockets. In addition, the platform also supports AMD's Milan-X chips (BIOS version 2.3 or newer) that come with up to 64 cores and 128 threads paired with a once-unthinkable 768MB of L3 cache. These chips help gear the 1024US-TRT for more diverse workloads beyond its traditional target markets, expanding to technical computing workloads, too. This includes workloads like Electronic Design Automation (EDA), Computational Fluid Dynamics (CFD), Finite Element Analysis (FEM), and structural analysis.

The 1024US-TRT has a tool-less rail mounting system with square pegs that eases installation into server racks, and the CSE-819UTS-R1K02P-A chassis measures 1.7 x 17.2 x 29 inches and slides into 19" racks.

The server accommodates CPU TDPs that stretch up to 280W, but using chips beyond 225W requires special accommodations. As such, the server can technically support the most powerful EPYC processors, like the 7Hxx-series models, but you'll need to verify those configurations with Supermicro. 

The front panel comes with standard indicator lights, like a color-coded information light that indicates various types of failures and overheating while also serving as a unit identification LED. It also includes hard drive activity, system power, and two LAN activity LEDs. Power, reset, and unit identification (UID) buttons are also present at the upper right of the front panel, with the latter illuminating a light on the rear of the server for easy location of the unit in a packed rack.

By default, the system has four tool-less 3.5-inch hot-swap SATA 3 drive bays, but you can configure the server to accept four NVMe drives on the front panel (you use 2.5" adaptors for the SSDs). We tested with a PCIe 4.0 Kioxia 1.92TB KCD6XLUL1T92 SSD. You can also add an optional SAS card to enable support for SAS storage devices and an optical drive. The front of the system also houses a slide-out service/asset tag identifier card to the upper left. This is important as it holds the default BMC user ID and password, enabling access to the remote management features. 

Popping the top off the chassis reveals two housings that hold fans. A total of eight fans feed air to the system, and each housing includes four Sunon 23,300 RPM counter-rotating 40 x 40 x 56 mm fans for maximum static pressure and reduced vibration. As expected with servers intended for 24/7 operation, the system can continue to function in the event of a fan failure. However, the remainder of the fans will automatically run at full speed if the system detects a failure. Naturally, these fans are loud, but that's not a concern. You manage the fan speed and profiles via the BMC (not the BIOS).

Four fans cool each CPU, and a simple black plastic shroud directs air to the heatsinks underneath. Dual SP3 sockets house the two processors, which are covered by standard CPU heatsinks optimized for linear airflow. 

A total of 16 memory slots flank each processor, for a total of 32 slots that support up to an incredible 8TB of ECC DDR4-3200 memory (via 256GB DIMMs), easily outstripping the memory capacity available with competing Intel platforms. We tested the EPYC Milan processor with 16x 16GB DDR4-3200 SK hynix modules for a total memory capacity of 256GB. In contrast, the Icel Lake Xeon comparison platform came with 16x 32GB SK hynix ECC DDR4-3200 for a total capacity of 512GB of memory. 

The H12DSU-iN motherboard's expansion slots consist of two full-height 9.5-inch PCIe 4.0 x16 slots and one low-profile PCIe 4.0 x16 slot, all mounted on riser cards. An additional internal PCIe 4.0 x16 slot is also available, but this slot only accepts proprietary Supermicro cards.

The rear I/O panel includes two 10 gigabit RJ45 LAN ports powered by an Intel X710-AT2 NIC, along with a dedicated RJ45 IPMI LAN port for management. Here we find the only USB ports on the machine, which come in the form of two USB 3.0 headers (it's a pity there isn't a USB port on the front), along with a COM and VGA port. 

Two 1000W 80% Plus Titanium-Level redundant power supplies with PMBus provide power to the server, with automatic failover in the event of a failure and hot-swapability for easy servicing. 

The BIOS is easy to access and use and offers plenty of tunable parameters, including CPU power threshold adjustments, while the IPMI web interface provides a wealth of monitoring capabilities and easy remote management that matches the type of functionality available with other types of platforms. Among many options, you can update the BIOS, use the KVM-over-LAN remote console, monitor power consumption, access health event logs, monitor and adjust fan speeds, and monitor the CPU, DIMM, and chipset temperatures and voltages. Supermicro's remote management suite is polished and easy to use, and we can say the same about the BIOS.

Test Setup

Bear in mind that the 240W 64-core EPYC 7713s we're testing aren't the highest-frequency 64-core in AMD's EPYC Milan arsenal. That distinction falls to the 64-core EPYC 7763 with a 2.45 base and 3.5 GHz boost clock rate, along with a much heftier 280W TDP. Keep that in mind when you compare this EPYC chip to the Ice Lake flagship, the Xeon Platinum 8380, which also comes with a much higher 270W TDP. 

We used the Supermicro 1024US-TRT server to test the EPYC 7713 Milan processors, while we used the Supermicro 1024US-TR4 server to test four different EPYC Rome configurations.

We used the Intel 2U Server System S2W3SIL4Q Software Development Platform with the Coyote Pass server board to test the Ice Lake Xeon Platinum 8380 processors. This system is designed primarily for validation purposes, so it doesn't have too many noteworthy features. However, the system is heavily optimized for airflow, with the eight 2.5" storage bays flanked by large empty bays that allow for plenty of air intake. You can read more about this configuration in our Intel Ice Lake Xeon Platinum 8380 Review. We used a Dell/EMC PowerEdge R460 server to test the other Xeon processors in our test group.

Our test configurations don't have balanced memory capacities, but that comes as an unavoidable side effect of the capabilities of each platform and the systems we've been sampled. As such, bear in mind that memory capacity disparities may impact the results below.  

We used the Phoronix Test Suite for testing. This automated test suite simplifies running complex benchmarks in the Linux environment. The test suite is maintained by Phoronix, and it installs all needed dependencies, and the test library includes 450 benchmarks and 100 test suites (and counting). Phoronix also maintains openbenchmarking.org, which is an online repository for uploading test results into a centralized database. We used Ubuntu 20.04 LTS and the default Phoronix test configurations with the GCC compiler for all tests below. We also tested both platforms with all available security mitigations. 

Linux Kernel and LLVM Compilation Benchmarks

The dual EPYC 7713 processors in the 1024US-TRT complete the Timed Linux Kernel Compilation benchmark, which builds the Linux kernel at default settings, in a mere 19 seconds, edging out the Xeon Platinum 8380s. That's incredibly impressive — remember, the 7713s operate at a 225W TDP, meaning it is the lower-frequency 64-core EPYC variant. In contrast, the Ice Lake chips have a 270W TDP rating. AMD's Milan flagship, the 7763, would take the lead in this benchmark. 

AMD's EPYC 7713 also carves out a win against the 8380s in the Timed LLVM Compilation workload, completing the test in 178 seconds. Additionally, performance scaling is more pronounced than we expected moving from the EPYC 7742 to the 7713 given that the latter is the low-frequency variant, with the 7713s completing the benchmark 10% faster than the previous-gen flagship. 

Molecular Dynamics and Parallel Compute Benchmarks

NAMD is a parallel molecular dynamics code designed to scale well with additional compute resources; it scales up to 500,000 cores and is one of the premier benchmarks used to quantify performance with simulation code. The EPYC processors are well-suited for these types of highly-parallelized workloads due to their prodigious core counts, but Intel's Ice Lake is also a force to be reckoned with. This workload fully saturates the cores with an extended workload, thus generating a heavy thermal load, but the Supermicro 1024US-TRT didn't show any signs of thermals impacting performance. Once again, the EPYC 7713 impresses against Intel's much higher-TDP flagship.

Stockfish is a chess engine designed for the utmost in scalability across increased core counts — it can scale up to 512 threads. Here we can see that this massively parallel code scales incredibly well, with the EPYC 7713s taking the top of the benchmark charts.   

Rendering Benchmarks

Provided you can keep the cores fed with data, most modern rendering applications also fully utilize the compute resources. The 64-core EPYC Milan and EPYC Rome processors lead this series of benchmarks convincingly, though Intel's Ice Lake does narrow the competition in a few of the Blender renders and leads in others.

Encoders tend to present a different challenge: As we can see with the VP9 libvpx benchmark, they often don't scale well with increased core counts. Instead, they often benefit from per-core performance and other factors, like cache capacity.  Here we can see that Intel's 8380 takes the lead by slim margins, but AMD's EPYC Milan 7713 is incredibly impressive given its much lower power and thermal budget. 

Compression and Security

Compression workloads also come in many flavors. For example, the 7-Zip (p7zip) benchmark exposes the heights of theoretical compression performance because it runs directly from main memory, allowing both memory throughput and core counts to impact performance heavily. Here we can see that benefit the EPYC 7713s tremendously as they take the lead over the 8380s. In contrast, the gzip benchmark, which compresses two copies of the Linux 4.13 kernel source tree, tends to respond well to speedy clock rates, giving the 16-core EPYC 7F52s the lead. The 7713s are still impressive, though, as they take second place. 

The open-source OpenSSL toolkit uses SSL and TLS protocols to measure RSA 4096-bit performance. As we can see, this test favors the EPYC processors due to its parallelized nature, but offloading this type of workload to dedicated accelerators is becoming more common for environments with heavy requirements.  

Conclusion

AMD's forward-thinking SP3 socket design has given server builders plenty of flexibility as the EPYC lineup matures, and that capability is on full display with the Supermicro 1024US-TRT server. With support for EPYC 7002 and 7003 processors, high-frequency H- and F-Series, and even the Milan-X CPUs, the platform has continued to evolve to support the growing EPYC roster. 

Supermicro's fully-validated systems come with the hardware fully configured, and rack installation with the tool-less rail kit is simple and quick. We found the system to be robust during our tests, with plenty of power for the processors fed by the high-quality Titanium power supplies and ample cooling provided by eight fans. We didn't notice any adverse impact from thermal generation during a bevy of extended-duration all-core workloads that generate plenty of thermal load, signifying that the cooling system is well designed. 

We'd like to see a few USB ports added to the front panel, as this eases setup and maintenance. That said, the easily-accessible IPMI interface provides comprehensive monitoring and maintenance options, not to mention a polished remote management interface. On the hardware side, the dual 10GbE LAN and PCIe 4.0 interfaces provide plenty of connectivity options, while the support for up to 8TB of ECC memory allows the server to exploit EPYC Milan's tremendous throughput while providing enough capacity to address even the most memory-hungry applications. The server's slim 1U chassis does limit you to four drive bays, but Supermicro supports plenty of high-capacity storage options to maximize the available bays.

The Supermicro 1024US-TRT server performed well in our tests, packing quite the threaded heft into a small form factor that can address general-purpose, enterprise, cloud, and virtualization roles well. The addition of Milan-X support also positions the platform for possible use in more diverse technical computing workloads, positioning the 1024US-TRT as a robust server fit for a wide variety of roles. 

• MORE: CPU Benchmark Hierarchy
• MORE: AMD vs Intel
• MORE: How to Overclock a CPU

Nvidia's decision to build its own lineup of Arm-based server CPUs, dubbed the Grace CPU Superchip and the Grace Hopper Superchip, charted the company's course to building out full systems with its CPUs and GPUs in the same box. That initiative moved closer to reality today as Nvidia announced at Computex 2022 that several major server OEMs will offer dozens of reference systems based on its new Arm CPUs and Hopper GPUs in the first half of 2023.

Nvidia tells us that these new systems, which we break down below, will co-exist with its existing lineup of reference servers, so the company will continue to support x86 processors from AMD and Intel for the foreseeable future. 

As a quick reminder, Nvidia's Grace CPU Superchip is the company's first CPU-only Arm chip designed for the data center and comes as two chips on one motherboard, while the Grace Hopper Superchip combines a Hopper GPU and the Grace CPU on the same board. The Neoverse-based CPUs support the Arm v9 instruction set and systems come with two chips fused together with Nvidia's newly branded NVLink-C2C interconnect tech.

Overall, Nvidia claims the Grace CPU Superchip will be the fastest processor on the market when it ships in early 2023 for a wide range of applications, like hyperscale computing, data analytics, and scientific computing. (You can read the deep-dive details about the silicon here.)

A slew of blue-chip OEM/ODMs, like Asus, Gigabyte, Supermicro, QCT, Wiwynn, and Foxconn, have dozens of new reference server designs planned for launch in the first half of 2023, indicating that Nvidia remains on track with its Grace CPU silicon. The OEMs will craft each of the server designs from one of Nvidia's four reference designs that include server and baseboard blueprints. These servers will be available in 1U and 2U form factors, with the former requiring liquid cooling.  

The Nvidia CGX system for cloud graphics and gaming applications comes with the dual-CPU Grace Superchip paired with Nvidia's A16 GPUs. The Nvidia OVX servers are designed for digital twin and omniverse applications and also come with the dual-CPU Grace, but they allow for more flexible pairings with many different Nvidia GPU models. 

The Nvidia HGX platform comes in two flavors. The first is designed for HPC workloads and only comes with the dual-CPU Grace, no GPUs, and OEM-defined I/O options. Meanwhile, on the far right, we see the more full-featured HGX system for AI training, inference, and HPC workloads with the Grace CPU + Hopper GPU Superchip, OEM-defined I/O, and support for the option of fourth-gen NVLink for connections outside of the server via NVLink switches.   

Notably, Nvidia will offer the NVLink option with its CPU+GPU Grace Hopper Superchip models, but not for systems powered by the dual-CPU Grace Superchip. 

Here we can see the two different 2U blades that can power the HGX systems. The dual-CPU 'HGX Grace' CPU Superchip blade has up to 1TB of LPDDR5x memory, provides up to 1TB/s of memory bandwidth, adheres to a 500W TDP envelope, can be cooled with either liquid or air, and supports two blades per node for up to 84 nodes per rack. 

The HGX 'Grace Hopper' Superchip blade comes with a single Grace CPU paired with the Hopper GPU, providing 512 GB of LPDDR5x memory, 80GB of HBM3, and up to a combined 3.5 TB/s of memory throughput. As you'd expect, given the addition of the GPU, this blade comes with a higher 1000W TDP envelope and comes with either air or liquid cooling. This larger blade limits the HGX Grace Hopper systems to 42 nodes per rack. 

Unsurprisingly, Nvidia offers all of these systems with its other important additive that is helping the company become a solutions provider: The 400 Gbps Bluefield-3 Data Processing Units (DPUs) that come as the fruits of its Mellanox acquisition. These chips offload critical work from the CPUs, allowing streamlined networking, security, storage, and virtualization/orchestration features.

Nvidia already has CGX, OVX, and HGX systems available with x86 CPUs from both Intel and AMD, and the company tells us that it plans to continue to provide those servers and develop newer revisions with Intel and AMD silicon.

"x86 is a very important CPU that is pretty much all of the market of Nvidia GPUs today. We'll continue to support x86, and we'll continue to support Arm-based CPUs, offering our customers in the market the choice of wherever they want to deploy accelerated computing," Paresh Kharya, Nvidia's Senior Director of Product Management and Marketing, told us. 

That doesn't mean that Nvidia will pull performance punches, though. The company recently demoed its Grace CPU in a weather forecasting head-to-head with Intel's Ice Lake, claiming that its Arm chip is 2X faster and 2.3X more efficient. Those same Intel chips currently power Nvidia's OVX servers. 

The company hasn't spared AMD, either. Nvidia also claims that its Grace CPU Superchip is 1.5X faster in the SPECrate_2017_int_base benchmark than the two previous-gen 64-core EPYC Rome 7742 processors it uses in its current DGX A100 systems.

We'll soon see if Nvidia's first foray into CPUs lives up to its claims. The reference CGX, OVX, and HGX systems will ship in the first half of 2023. 

Multiple European retailers have begun listing Supermicro's MBD-X13SAE-series motherboards based on Intel's yet-to-be-announced W680 chipset. That chipset is designed for upcoming Intel's Xeon E-3300-series 'Alder Lake' CPUs in LGA1700 packaging. The listings imply that Intel's next-generation entry-level Xeon E platform for entry-level servers and workstations is nearing its launch. But some retailers will sell you the motherboards today.

Supermicro's MBD-X13SAE-series family will include at least two Intel W680-based models: the MBD-X13SAE and the MBD-X13SAE-F, according to a listing at Sona.de (spotted by Twitter leaker @momomo_us). Both motherboards support all of the processors in Intel's LGA1700 form-factor, including the unannounced Xeon E-3300-series CPUs featuring the hybrid Alder Lake architecture, and are equipped with four memory slots supporting DDR5 memory modules with or without ECC (according to Ahead-IT.eu). Also, the new platforms naturally support PCIe 5.0 as well as other advantages that Alder Lake has to offer. 

Traditionally, memory link ECC (do not confuse with DDR5's on-die ECC technology that is implemented to improve DRAM IC yields) is one of the key features that differentiates Intel's Xeon E-series CPUs that support memory modules with from similarly configured Core processors that do not support ECC capability. That said, Intel's E-3300-series CPUs and Intel's W680 platform will require DDR5 memory modules with ECC to show one of its key advantages over desktop-oriented platforms. 

But while the Intel's Xeon E-3300-series processors and Intel's W680 platform are getting closer, we still have little idea about what to expect from new CPUs and chipset. Early leaks indicated that the new entry-level server/workstation platform will offer up to 16 CPU cores with TDPs of 35 W, 65 W and 125 W. Meanwhile, we have no idea whether Intel intends to enable AVX-512 support on its upcoming Xeon E-3300-series processors, or will keep it disabled, like it does with 12th Generation Core CPUs. 

As far as availability is concerned, Supermicro's MBD-X13SAE-series motherboards based on Intel's W680 chipset can be ordered today from several European retailers. Two Intel W680-based platforms — the MBD-X13SAE (priced at €442 with VAT or $425 without tax) and the MBD-X13SAE-F (priced at €492 with VAT or $473 without tax) — can be ordered from Sona.de or CTT.de and delivered in 3 – 5 business days. The Netherlands-based Ahead-IT.EU promises to ship these motherboards in 5 – 7 days, whereas Italy-based IsyCorp says that these products are available from the manufacturer. 

Given that so many retailers are about to ship their Intel W680 motherboards, it looks like the launch of Intel's next-generation Xeon E-3300-series platform is imminent.

In conjunction with Intel’s Alder Lake CPU announcements, motherboard partners have pulled back the curtain on dozens of new Z690-based motherboards to go along with the upcoming 12th Gen chips. The new motherboards support the latest and greatest technologies from the new processor, including a bump to PCIe DDR5 (some motherboards have DDR4 support, more on that later), PCIe 5.0 and upgraded power delivery. Although there’s a lot to be excited about hardware-wise, what I’m looking forward to most are the new board designs. We’ll review many of these over the coming months, and some will be good enough to make our best motherboards page.

For now, we’ve comprised a list of motherboards with as much information directly from the partners as possible. That said, many details  (including pricing) will arrive later, so we’ll provide what we have and update the information here as we get it.

Intel Z690 Chipset: Different Socket, Similar (but Upgraded) Features 

At the time of this writing, Intel has not released the full details of the Z690 chipset. However, we do know a few things that differentiate Z690 from the previous-gen Z590. Unlike Z490 and Z590, Z690 uses a different socket with more pins, dubbed LGA1700. The new socket is not compatible with LGA12/15xx, so any heatsink, AIO, or custom block requires new mounting hardware. Many cooling companies, including (but not limited to) be quiet!, Alphacool, MSI and Noctua are offering free upgrade kits. Be sure to reach out to whoever made your cooler and ask about that if you plan on purchasing the new platform and using your existing cooling solution. 

In addition to the flagship Z690 chipset, Intel is also releasing lesser H670, B660 and H610 chipsets that cater to budget-conscious or business users, with the latter set as entry-level. Like the 500 series, we expect the H670 and B560 to overclock memory but not the CPU. For those waiting for an HEDT update, you're not forgotten, as the X699 chipset appears to be on the horizon. Workstation users will get the W680 and W685 for Xeon processors.

Like Comet Lake, Alder Lake supports a total of 20 lanes from the CPU itself. In total, there are 16 PCIe 5.0 lanes and four additional PCIe 4.0 lanes. The platform divides the 5.0 lanes as x16 or x8 for graphics and x4/x4 for storage, enabling a full 64 GBps bandwidth. The four PCIe 4.0 lanes provide additional storage connectivity. 

Outside of the new socket, one of the most significant differences in the new Z690 chipset is native PCIe 5.0 support, which doubles the available bandwidth to 64 GBps. While we’re a ways away from PCIe 5.0 hardware, and even further from when that type of throughput is needed, the bandwidth is here. AMD was first to market with PCIe 4.0; now it’s Intel’s turn to lead the bleeding edge with PCIe 5.0. With all of this available bandwidth, you can run more devices (M.2 slots, for example) off the CPU-connected PCIe lanes and not lose performance from your graphics card. There are also faster lanes on the chipset, allowing devices to use fewer lanes to achieve their full performance, increasing board makers' flexibility with where to attach items.

The DMI link between the chipset and CPU has also increased. Z590 doubled the link speed, moving from PCIe 3.0 to x4 to PCIe 3.0 x8. With Z690, Intel is doubling that again, but this time the DMI link jumps up to PCIe 4.0 x8. The increase doubles the bandwidth available for any chipset-connected devices (networking and storage, for example) and allows users to run more attached devices concurrently without losing performance.

DDR5 also makes its consumer desktop debut here, offering users increased bandwidth and capacity along with lower power consumption. Alder Lake sports a 128-bit memory bus, so four 32-bit DDR5 channels (DDR5 = two 32-bit channels /module). The base voltage drops to 1.1V for JEDEC specs instead of the 1.2V we’re used to with DDR4. High-performance kits will still be at 1.2V or above. For those who don’t want to spend the premium to invest in DDR5, you’re in luck. 

Surprisingly, there are Z690 boards that only support DDR4. Rumors suggest that, currently, the performance differences between DDR4 and DDR5 aren’t significant in many use cases. However, we don’t expect much difference until DDR5 matures (speeds increase and latencies decrease). But we’ll investigate and report back once we have the boards in hand.

While there are other distinctions, most are more subtle. Here’s a look at how Z690 compares with Intel’s previous mainstream flagship chipsets. Keep in mind some details are still uncertain as of this writing.

The networking situation doesn’t change too much on Z590. The chipset continues integrated support for Intel CNVi Wi-Fi, but upgrades to Wi-Fi 6E support as the maximum supported, versus Wi-Fi 6 on the previous platform. This upgrade won’t matter for most users, as to run 6E speeds you need to have a router capable of 6E and an internet service provider that offers speeds to take advantage of it. For much more on this, see our Wi-Fi 6 and 6E explainer feature. Like Z590, board partners need to just add the Wi-Fi card to the board for support. 

You’ll find 2.5 Gb NICs on most Z690 boards on the wired front, while some flagships go higher with 5 GbE or 10 GbE. Some budget boards will still use 1 GbE, which is still acceptable for most users. Since a majority don’t have 1 Gb-plus internet service in the first place, the faster ports are still valuable for transfers within your network (assuming the slowest part is up to the task).

We know that USB 3.2 Gen2x2 (20 Gbps) is again supported natively. The chipset diagram describes up to four 3.2 Gen 2x2 ports, 10x 3.2 Gen 2, 10x 3.2 Gen 1 and finally, 14 USB 2.0 ports. There’s obviously not enough physical space for the maximum for each, so each board will have a different count.

SATA3 6 Gbps ports also get a bump up to eight total (from six). Any motherboards with more than eight ports will use a third-party controller, typically from ASMedia. As mentioned earlier, the most significant differences are the PCIe 5.0 support from the CPU and DDR5. Intel has caught up to and surpassed AMD on that front, at least with CPU-connected lanes. While PCIe 5.0 video cards don’t exist in the desktop market yet, the increased bandwidth means other attached items can use fewer lanes and function as fast as possible, minimizing lane sharing and the resultant disabling of ports as you plug more devices in.

While Z690 has arrived, we still dont’ know exactly how the CPUs that will drop into them will perform. Rumors have the new Golden Cove cores sporting a 19% IPC increase over Cypress Lake (Rocket Lake’s microarchitecture), which should best AMD’s Zen2/Ryzen 5000 series processors. But we’ll have to wait for our own testing to say anything for sure on that front. But at the time of this writing, there are over 60 Z690 boards to choose from. So if you’re looking to build a new PC based on Intel’s latest, there should be something for everyone. 

Alder Lake CPUs 

Alder Lake is built on the Intel 7 process node technology and is the first to use a hybrid architecture in the desktop space. The hybrid design in Intel’s terms includes having a set of high ‘Performance’ cores coupled with high ‘Efficiency’ cores -- or P-core and E-core as Intel calls them. The new Performance cores are based on the new Golden Cove microarchitecture, with the Efficiency cores based on the Gracemont architecture. The theory is that the P-cores will handle single-threaded tasks that require low latency, while the E-cores will be better in multi-threaded or power-limited situations. It's worth noting that hyperthreading still makes its way into these CPUs, but only on the P-cores and only on specific processors..

As far as the list of processors goes, six unlocked desktop variants will be available at launch, starting with the flagship Core i9-12900K. The new top-end unlocked processor is a 16-core (8P+ 8E) / 24 thread part with a base clock of 3.2 GHz boosting to 5.2 GHz. The budget end sits with the i5-12600KF, a 10-core (6P+4E) 16 thread part. Refer to the table and image below for a list of the processors announced so far, as well as other high-level information.

Where Rocket Lake went with fewer cores/threads than the previous-generation Comet Lake, Alder Lake processors increase their count over Rocket Lake. Comparing flagship-to-flagship, the new i9-12900K doubles the core count and has a 33% increase in total threads available. Couple that with the expected Instructions Per Clock (IPC) increase and, on paper, you should have a faster processor compared to previous-gen parts. 

The flagship chips have a TDP of 125W for base power and a maximum turbo power (PL2) of 241W. What we do know is that Alder Lake uses more energy than Comet Lake. With that, I do wonder what cooling needs are like, especially when overclocked. To that end, Intel has made adjustments to the CPU itself by thinning out the die and the sTIM layer, while increasing the thickness of the integrated heat spreader (IHS). To support the new processors, board partners have raised the bar on VRM and VRM cooling in Z690. Flagship-class and high-end motherboards step up to 105A MOSFETs, while mid-range and budget boards also get a bump to keep these processors going.

Memory support for the new CPUs also gets a considerable bump from DDR4-3200 to DDR5-4800 when using the new Alder Lake-based processors. At the time of this writing, we didn’t have details on memory support for Z690 boards, but you can expect it to be well over the base spec. We’ve seen some supporting well over DDR5-6000. As usual, your mileage will vary, and working with the correct CPU (with an excellent integrated memory controller), memory kit, and motherboard choice are critical when chasing high memory clocks. That said, the sweet spot for memory speeds and performance per dollar changes as the process matures. Speeds will go up while CAS rating/timings, in general, will go down. For now, it seems like somewhere in the DDR5 5200-5600 range is likely where the sweet spot will be, at least initially. We do know that DDR5 pricing is going to be notably higher than DDR4. So it goes.

Z690 Motherboards: The Full List (So Far) 

With the chipset details out of the way, we’ve provided a list of all Z690 motherboards announced below, doing our best to ignore rumors and unsubstantiated leaks. However, several board partners didn’t respond to inquiries in time for publication, so there's a strong chance that some things will change. Board partners provided the limited information listed in the following tables. We’ll break things out on the following pages and offer some thoughts on the respective launch lineups after. Unfortunately, pricing is an essential piece of information that no vendors have provided yet, but we’ve filled in the tables with what data we have. 

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

ASRock: 20 boards

• Improved aesthetics and power delivery
• 2.5GbE LAN on most (all?) SKUs
• Wi-Fi 6(6E) on some boards

At the time we wrote the article, ASRock had yet to respond to our inquiries about their lineup, so we're relying more on rumors than we'd like. That said, we fully expect ASRock to come out of the gate sporting a full lineup of motherboards to choose from. The information below is sourced from the web in regards to the  count and names.

From the images we do have, you’ll notice the appearance of the boards has changed. At a high level, we expect to see PCIe 5.0 and DDR5 support, more powerful VRMs, updated audio codecs, and an increase in M.2 storage count on most boards. ASRock, like the other board partners, will also have boards with DDR4 options available. 

Familiar names make an appearance again in Z690. They include the Z690 Taichi, PG Velocita, Extreme, Phantom Gaming 4, Pro and more. There are mATX- and Mini-ITX-sized boards, along with entry-level and halo products. But again, with so few details, there are plenty of question marks in the table below.

Asus: 14 boards 

• Improved aesthetics and power delivery
• Teamed Power Architecture (on most boards)
• AI Overclocking Based on CPU quality and Cooler Efficiency
• 2.5GbE and 10GbE NIC Available, along with Wi-Fi 6E

Asus’ initial Z690 lineup consists of eight boards, with the rest likely filling out in the weeks and months to come. Out of the gate are samples from each of the company’s internal market segments, including the ROG Maximus (XIV Extreme/Hero/Apex), ROG Strix (Z690-E Gaming), TUF Gaming Z690-Plus Wi-Fi, Prime Z690-A and the ProArt Z690 Creator.

The premium gaming and overclocking-focused ROG Maximus XIV series continues to bring the best of what the platform has to offer from Asus. Large passive heatsinks help cool the teamed 105A VRMs underneath, save for the Glacier, which includes a water block for the CPU and VRMs.

Asus includes AI overclocking from the BIOS or the AI Suite, which tests the processor to see which cores are best. Like some other brands, CPU quality and efficiency determine the outcome. All Maximus boards include at least a 2.5 Gb LAN, with the Extreme using Marvell AQtion AQC107 10GbE. You’ll also find up to eight SATA ports and five M.2 slots in this lineup.

Historically, the ROG Strix series spans several boards of various sizes, including ATX, Micro ATX, and Mini-ITX. All employ a capable VRM design, but details are scarce across this entire lineup. Many boards receive the updated Supreme FX Realtek 4080 audio codec, though. 

The Asus Prime motherboards have expanded significantly for Z690, with five options including an mATX option. These Prime boards offer users a less-expensive way into the platform without sacrificing too many features and offer a unique black and grey/silver design aesthetic. 

The TUF series boards are geared more toward gamers with more modest budgets and needs. Users still get capable power delivery and heatsinks, integrated Wi-Fi 6E on the Plus (Wi-Fi), along with an updated appearance and an overall increase in M.2 socket count. 

In all, Asus has boards of all shapes, sizes, and likely budgets to fit your needs. Although the company is launching with up to 14 boards, we’ve only listed what Asus provided below.

Biostar: 3 boards 

• Up to 20-phase design
• Wi-Fi 6 capabilities on all boards 

Biostar is releasing three Z590 based boards —  two that carry the Valkyrie name (Z690 Valkyrie and Z690A Valkyrie) and the Z690GTA on the budget side. The Z690 Valkyrie boards include a USB 3.2 Gen 2x2 (20 Gbps) Type-C port, four M.2 sockets, eight SATA ports, 2.5 GbE and Wi-Fi 6, and more. A 20-phase VRM, like the Z590 version(by count), should handle overclocking the flagship i9-12900K. 

The Z690GTA hails from the budget side and offers users a slew of USB ports (though not a 3.2 Gen 2x2 Type-C port), the same (last generation) Realtek ALC1220 audio codec as the Valkyrie, three M.2 sockets, and a less-powerful but still capable (17-phase) VRM. Styling on the Z690GTA has also changed, with this generation sporting a black-and-light-blue theme. Over the years, Biostar has continuously improved and we expect that to continue with Z690. We just hope the BIOS layout is updated, as that was one of the company’s shortcomings in the past.

EVGA: 2 boards

EVGA is another company we haven’t heard back from, but we expect that the EVGA Z690 product stack has two boards: the familiar Z690 Dark and Z690 FTW. The Dark is the flagship board in an E-ATX form factor, sporting a beefy VRM and other gaming and overclocking features. The FTW is also focused on the gamer and overclocker, but tends to be the less expensive but still a well-appointed option.

Details are non-existant here, but we expect the boards to receive similar updatesas the other Z590. boards with PCIe 5.0 and DDR5 support, an updated audio codec, updated aesthetics, and perhaps another M.2 socket or so. As usual, we expect these to be available well after launch. 

Gigabyte: 22 boards

• 60-105A VRMs
• Daisy Chain + Shielded Memory Routing
• USB 3.2 Gen 2x2 (20 Gbps) port for all Z690 boards
• 2.5 GbE-plus NICs on all models (up to 10 GbE)
• Front USB Type-C for all Z690 models
• New styling 

Gigabyte joins the Z690 fray with more than different options, including the gaming-centric Aorus line and the familiar Xtreme (with two PCIe 5.0 slots), Master, Ultra, Elite and Pro boards along with the more budget-focused Gaming X and UD series. The Vision (designed for creators) was replaced last year by the Aero line, which aslo has made its way to Z690. You’ll find all sizes and budget tiers, including several boards that run with DDR4 instead of DDR5. So if you want to save a bit of money and go with DDR4, you have that option with Gigabyte. If you’re jumping into DDR5, the new Z690 boards include Ultra Durable SMD DDR5 Armor and Shielded Memory Routing to get the cleanest signal to the chips. 

One of the most significant changes to these boards, outside of the different socket, are the VRMs. Power delivery ranges from 15 total phases using 60A parts to 23 phases and 105A parts at the high end. Most, if not all SKUs received an upgrade to support the new processors and their higher power draw. Gigabyte also updated some cooling methods, including Fins-Array III to cool the VRMs and Thermal Guard III, a heatpipe connected to fins for the top M.2 socket (Xtreme only). 

All boards include a USB 3.2 Gen 2x2 Type-C (20 Gbps) port, 2.5 GbE NIC, and most include Wi-Fi 6 or 6E (outside of some UD boards that use AC/AX Wi-Fi). Other features include the jump to next-gen Realtek 4000 series audio codec in some models and an increase in M.2 sockets, with many running up to four (Aorus Master has a whopping five!) thanks to the increased lanes and bandwidth from the CPU.

Like a few other board partners, there’s something for everyone in this deep product stack from Gigabyte.

MSI: 18 boards

• 70A to 105A VRMs
• 2.5 GbE LAN and Wi-Fi 6 or 6E for All Models
• Memory support up to DDR4 6800
• USB 3.2 Gen 2x2 (20 Gbps) Type-C port for all models
• New styling

MSI attacks Z690 with 18 SKUs covering both DDR5- and DDR4-compatible motherboards. The MEG/MPG/MAG lines (Gaming) and Pro lines (for Creators) all make a return. At the top of the stack, the MEG boards all use a direct power arrangement, with the MEG boards all using 105A SPS MOSFETs. All MEG and MPG boards use Direct Power (no phase-doublers), while the MAG and Pro series use a “Duet Rail” power system (DRPS). In the end, all of MSI’s Z690 boards use an upgraded VRM to support the Alder Lake processors better.

In addition to the power delivery improvements and aesthetic changes, a vast majority of MSI boards support four M.2 connectors (some even support five), also an upgrade from the last generation. Any board that comes with the M.2 Xpander card also gets PCIe 5.0 support for the storage module. It's worth noting is MSI uses an EZ M.2 Clip to help install/remove M.2 SSDs quickly without fumbling around with tiny screws. Other popular features include the M.2 Shield Frozr to keep the M.2 modules cool, and heatpipe-cooled VRMs on a lot of the lineup.

On the networking front, unlike other brands, the existing MSI lineup doesn’t have a 10 GbE NIC on its flagship or any other board. However, you do find 2.5 GbE NICs on all boards and Wi-Fi 6E on mos models. All boards include at least one USB 3.2 Gen 2x2 (20 Gbps) Type-C port and support many others. SATA port count across the Z690 stack varies from four to six depending on the SKU. Between the increase in M.2 count and SATA ports, nobody should lack storage options in this generation. Like other vendors, MSI uses the latest Realtek audio codec (4000 series) across most of its line, with last-generation’s 1200 series found on others.

The venerable Godlike is missing from the list.. But we hope to see this board soon, as previous versions have been monsters. Rest assured, the rest of the fan favorites are here, including the budget-friendly MAG Z690 Tomahawk and Torpedo boards, as well as overclocking-centric Unify-X and a Mini-ITX board. There’s something for everyone here, too.

NZXT: 2 boards

We reached out to NZXT for information, but they also didn’t respond in time for publication. That said, we expect to see them in the market again with an N7 Z690 SKU. In the past, the NZXT boards were based on an ASRock part, and we don’t expect that to change. There are rumors of a more budget-focused motherboard, dubbed the N5 Z690, that will also hit the market. But, again, those are rumors and not confirmed. 

Supermicro

We also reached out to Supermicro to see what the company has cooking for Z690. Like NZXT and ASRock, we didn’t hear back in time for this article, but we expect to see a SuperO board on this platform eventually. The server-like motherboards from SuperO are typically well built and option-heavy, but its server roots (aesthetics and BIOS layout) can be a turnoff for some. 

That’s it for now, though there’s plenty on offer already for Z690. After a couple of years of trailing AMD with technology and performance, the addition of PCIe 5.0 and DDR5 support, along with other improvements, has Intel leading the mainstream motherboard charge once again. As noted earlier, we’ll fill in some of the question marks above as we get more info and add in pricing as soon as we have it.

Time will tell whether the claimed 19% IPC increase for Alder Lake-S CPUs and hybrid architecture will help Intel retake the gaming crown and offset the BIG.little core configuration compared to AMD. One thing is clear though: Intel’s board partners are ready and waiting to give Alder Lake-S the support it needs to get the most out of the company’s new processors. If only some of these companies could keep their recent graphics cards in stock at reasonable prices to drop into these new builds, the end of 2021 might be an interesting time to build a new PC.  

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Supermicro is best known for server hardware, but the company does occasionally dabble in the consumer-focused space. Its latest offering on that front is the SuperO C9Z590-CGW, a board based around Intel’s flagship mainstream chipset. The motherboard (which is still basically server-class) includes native PCIe 4.0 support (with Rocket Lake CPU), a Marvell 10 GbE NIC and two USB 3.2 Gen2x2 Type-C ports. There’s three M.2 slots, though not much else has changed on this RGB-free board compared to the previous-gen model. At $429.99, the CGW (the CG, without Wi-Fi, is $409.99) is a full-featured Z590-based motherboard looking to make a place for itself in a crowded premium segment.

Being a board from Supermicro, it’s not exactly shocking that function trumps aesthetics here. To that end, its appearance isn’t quite up to the heavily shrouded look of similarly competing alternatives. Although there are heatsinks on the M.2 sockets, you don’t find any shrouds or RGB LEDs. That isn’t bad for some, but those looking for any kind of RGB lightning need to look elsewhere.

When talking about performance, this board stuck closely to the Intel specification out of the box, and we had to run our memory at DDR4 3200 (the maximum stock speed for the processor). Due to that, results were a bit lackluster in some areas. To get the most out of your CPU, you may need to unlock some power and current limits, but memory overclocking is still limited. Chances are if you’re looking at this motherboard at all, you aren’t looking to overclock your RAM. The board is built for stability. Overall, it’s a well-equipped board and one of the least expensive with a 10 GbE NIC, but that only goes so far. We’ll cover those features and more on the following pages. First, here are the full specifications from Supermicro.

Specifications - SuperO C9Z590-CGW

The box includes most of the required items to get you started but the accessory pack isn’t the most comprehensive we’ve seen. Below is a complete list of all included extras.

• Quick Reference Guide
• Driver disk
• Wi-Fi Antenna
• (4) SATA cables
• (3) M.2 screws
• IO Plate
• Q-connector
• Labels (for storage)

Removing the motherboard from the retail box, we see a semi-gloss black PCB with black and grey heatsinks and shrouds sporting a brushed-aluminum finish. Both the full-length PCIe slots and the four DRAM slots are reinforced to prevent shearing and EMI protection. The SuperO name is on the IO cover, with a frosted white stripe illuminated from below. That’s the only aesthetic lighting element on the board.

Curiously, the C9Z590-CGW doesn’t even have RGB headers, in case you wanted to add your own lighting. While the board has DNA from the server world, it’s marketed as a gaming motherboard and should include RGB headers. If you look at other boards around the $400 mark, integrated RGB lighting (plus additional headers) is everywhere, along with what many would consider a more premium (more shrouds and metal) appearance.

Starting with the top half of the board, the plastic IO cover reaches out over the black VRM heatsink on the left side. On top, above the socket, is a second small heatsink designed to cool the rest of the VRMs. Between the power delivery heatsinks along the top edge is a single 8-pin EPS connector to feed power to the CPU. Next to it is the first (of six) 4-pin PWM headers. You’ll need PWM-mode spinners to control your fans, while DC-controlled fans run at full speed. The manual doesn’t appear to list any values as far as output goes, so it’s best to assume a maximum of 1A/12W for each.

To the right of the socket area are the four reinforced, single-sided locking DRAM slots. The primary slots (when using two sticks) are grey, while the secondary slots are black -- a helpful visual cue. Supermicro lists memory speeds to DDR4 4000. However, we struggled with our GSkill DDR4 3600 kit and ran at 3200 MHz for testing. Perhaps with additional tweaking, we could have reached that mark, but XMP didn’t work, and we didn’t have luck spending some time trying to reach those speeds manually.

Along the right edge are three more 4-pin fan headers, along with convenient power and reset buttons. Further down the edge is the 24-pin ATX header to power the board and a front-panel USB 3.2 Gen2 Type-C header.

The SuperO C9Z590-CGW utilizes a 6+2+1+2 (Vcore/GT/SA/DIMM) configuration for Vcore, graphics, and system agent. The 8-pin EPS connector feeds power to the CPU and is then sent to an Infineon XDPE12284C (6+1?) controller directly to the six Infineon TDA21490 90A OptiMOS MOSFETs for Vcore. The 540A available for the CPU is low for this class (and price) of board, though it handled our Intel Core i9-11900K overclock (5.1 GHz) without issue.

Moving down to the bottom half of the board, starting with the left side, we see the audio bits fully exposed. Midway up is the Realtek ALC1220 codec, along with a few small audio caps. We also see the line that separates the audio parts from the rest of the motherboard. Just above that is a small black square heatsink that cools the Marvell 10 GbE chip underneath.

In the middle of the board are three M.2 sockets, each with its own individual heatsink. The top socket, M.2_M3, is connected to the CPU and runs up to PCIe 4.0 x4 speeds (64 Gbps, with a Rocket Lake-based CPU), supporting PCIe modules up to 80mm. The bottom two sockets source their bandwidth from the chipset and run at a maximum of PCIe 3.0 x4 (32 Gbps). The bottom two sockets support both SATA and PCIe-based M.2 devices.

Next, we spy the two reinforced full-length PCIe slots, along with the two x1 size slots. The two full-length slots are for the GPU, with their lanes sourced from the CPU. Both sockets are wired for x16 capability. When both are populated, they break down to x8/x8 configuration. Supermicro chose to forgo the PLX chip found on the Z490 board, dropping one of the rare ‘gaming’ boards with them. That said, this isn’t a huge concern as SLI and multi-GPU functionality is legacy at this point for mainstream use cases. The bottom two slots are connected through the chipset and run at PCIe 3.0 x1 speeds.

Continuing past the chipset heatsink to the right edge, we see another fan header and the four horizontal SATA ports. The SATA ports support RAID0, 1, 5 and 10 modes. In total, you can run up to three M.2 devices and four SATA devices -- plenty for most users. Below are the front-panel headers, along with the two-character debug LED to troubleshoot POST issues.

Across the bottom of the board are several headers, including additional USB, fan headers and more. Below is the complete list, from left to right:

• Front-panel audio
• Speaker header
• Intel Manuf. mode
• USB 2.0 header
• 5V Standby header
• USB 3.2 Gen1 header
• TPM header
• 4-LED debug
• COM port

Lastly, we spin around to the rear IO area. One of the first things you’ll notice here is that the C9Z590-CGW does not come with the rear IO plate pre-installed -- you’ll have to do it yourself, while other $400 boards have this installed from the factory. From left to right, we see the Displayport (1.4a) and HDMI (2.0b) ports. Next to that is a stack of four USB 2.0 ports. The red ethernet plug is for the 1 GbE Intel port, while the black is the Marvell 10 GbE. Below these are two USB 3.2 Gen2 Type-A ports and two ultra-fast USB 3.2 Gen2x2 Type-C ports. Continuing right, we spy a CMOS reset button, along with Wi-Fi card/connections and the 5-plug plus SPDIF audio stack.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Firmware

To give you a sense of the Firmware, we’ve gathered screenshots showing most BIOS screens.

The BIOS on these Supermicro boards do the job and have improved since they first came out. However, they still aren’t as refined as the others. It still feels like a server BIOS, in that the look is relatively simple. In EZ Mode, you’re able to get an idea of where the system stands and change a couple of options. Using the Advanced mode, you’ll see headers across the top (Main, Overclocking, Advanced, H/W Monitor, Boot, Save & Exit and BIOS Update) with options listed below.

Although the BIOS works just fine, POST takes a lot longer than most consumer-facing motherboards. The way your selection is highlighted works differently, so it takes a bit of getting used to. In essence, you have two things highlighted at once and have to hit enter to select the field to get it to move. Typically, the highlight follows the cursor without making a selection.

Software

Supermicro has the SuperOBooster application on the software front, an all-in-one monitoring and adjustment tool designed for its motherboards. The software is easy to read and maneuver around and controls the CPU and memory, fans, voltage, and updating the BIOS. SuperOBooster does a good job of presenting system information as well as being simple to use.

Test System

As of March 2021, we’ve updated our test system to Windows 10 64-bit OS (20H2) with all threat mitigations applied. On the hardware front, we’ve switched to all PCIe 4.0 components. We upgraded our video card to an Asus RTX 3070 TUF Gaming and the storage device to a 2TB Phison PS5-18-E18 M.2. We’ve also updated the games to F1 2020 and Far Cry: New Dawn, along with the hardware changes. We use the latest non-beta motherboard BIOS available to the public unless otherwise noted (typically during new platform launches). The hardware used is as follows:

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Benchmark Results

Our standard benchmarks and power tests are performed using the CPU’s stock frequencies (including stock Thermal Velocity Boost), with all power-saving features enabled. Optimized defaults are set in the BIOS and the memory set using the XMP profiles. For this baseline testing, Windows is set to the balanced power plan.

Synthetic Benchmarks

Synthetics are a great tool to determine if a board runs out of spec, as identical settings should produce similar performance results. Advanced memory timings are the one place where motherboard makers can still optimize for either stability or performance, though, and those settings can impact some testing.

Starting with the synthetics, overall the board performed much like the other boards that closely follow intel specifications. 7Zip compression was the highest result so far, but decompression was one of the slowest. Cinebench was also slower in both single and multi-threaded tests, as was POV-Ray. The PCMark 10 suite was all over the map, showing good and bad results. The AIDA benchmarks were quite low, but that was expected since we ran the sticks at DDR4 3200. Between a combination of slower RAM speeds and following the intel spec, the results averaged out similar to the other boards that follow the intel specification out of the box.

Timed Applications

LAME testing showed the SuperO C9Z590-CGW as one of the fastest results at 11.31 seconds. Corona and Handbrake results were the polar opposite, coming posting the slowest results so far. The fact that the board follows Intel’s specifications holds it back out of the box in long-running programs.

3D Games and 3DMark

We’ve recently updated our game tests to F1 2020 and Far Cry: New Dawn. The games are run at 1920x1080 resolution using the Ultra preset. As the resolution goes up, the CPU tends to have less of an impact. The goal with these settings is to determine if there are differences in performance at the most commonly used resolution with settings most people use or at least strive for.

In 3DMark Fire Strike, the 15,681 points scored was the second lowest of our Rocket Lake results, while Time Spy ended up scoring 13,432, which is average among the tested Z590 boards. The Far Cry: New Dawn result was the slowest we’ve seen by well over 10%, averaging 123 FPS against an average of over 140 FPS. F1 2020 results were also the slowest at 206 FPS. Here, however, the difference from fastest to slowest is only around 5%. It would be difficult to see the difference in this title without running the benchmark, but you would notice the difference in Far Cry.

Power Consumption / VRM Temperatures

We used AIDA64’s System Stability Test with Stress CPU, FPU and Cache enabled for power testing, using the peak power consumption value. With Rocket Lake, we need to disable AVX-512 instruction sets; otherwise, temperatures are out of control at stock speeds on some boards. The wattage reading is from the wall via a Kill-A-Watt meter to capture the entire ecosystem. The only variable that changes is the motherboard; all other parts are the same.

On the power consumption side, the SuperO C9Z590-CGW did well, especially on idle, where it used the least amount we’ve seen so far at 45W. Load wattage was about average, peaking at 277W during our test. This averages out to 161W, placing the board on the more efficient side of power use.

SuperO’s 90A MOSFETs worked well, efficiently managing our Intel Core i9-11900K CPU at both stock speeds and while overclocked. When running at stock speeds, the VRM peaked at 50 degrees Celsius. This was better than most of the several Z590 boards we’ve tested, though worse than the overclocking boards in the chart. When overclocked, the C9Z590-CGW kept the VRMs just under 65 degrees Celsius. This result was notably worse than overclocking-focused boards, but still well within operating parameters. We had no issues overclocking our processor using ambient cooling.

Overclocking

Overclocking on the SuperO board was straightforward. We set voltages, disabled and TVB boost(s), and any Adaptive boost, so the clocks hold and are unaffected by these automated features. The BIOS is laid out well, but the highlighting and selecting takes some getting used to. Outside of that, most overclocking options were on the surface, though some of the power options are found in different sections.

When overclocking the Core i9-11900K, we decided to do so by testing without AVX-512 instructions. When you unlock all of the power limits, you’re thermally limited, with the CPU reaching 100 degrees Celsius in several seconds. Since every board is different in following (or not following) Intel specifications, we’ve removed that variable. The bottom line is that if you need to use these instructions, you’ll need to set a significant offset compared to non-AVX-512 loads. To that end, we settled on a 5.1 GHz overclock at around 1.3V. Our CPU uses around 225W in this configuration, which lands between stock power use (195W) and AVX-512 power use (~265W) and is the end of the line due to thermal constraints.

Moving on to memory, here we ran into a few problems. To start, I was unable to get any (three total) kits of our GSkill memory to work with XMP. I tried to set DDR4 3600 speeds manually, but that was not successful either. Admittedly, I’m not the greatest with memory tweaking and overclocking, but we’re hitting a ‘sweet spot’ that’s just barely over max spec here. I’ve seen other reviews that were successful at DDR4 3600, so perhaps the kit(s) used was the issue. In short, if you want plug-and-play compatibility, buy a DDR4 3200 MHz kit--or a different motherboard.

Final Thoughts

Supermicro’s latest entry into the consumer gaming market improved in some ways and took some steps back in others. In addition to native Rocket Lake and PCIe 4.0 support, the C9Z590-CGW adds an M.2 socket (three total now), sports two USB 3.2 Gen2x2 Type-C ports, keeps the same VRM configuration and appearance, but removes the PLX chip for this generation.

Performance out of the box follows Intel specifications closely and, due to this, is overall one of the slower boards tested (alongside other boards that adhere to the spec). The low memory speed and timings didn’t help. If you adjust the turbo power limits so it doesn’t follow Intel specifications, the results are a lot closer to average.

As far as improvements are concerned, when using the Z490-based CGW, we were able to reach DDR4 4000 and couldn’t even run DDR4 3600 on C9Z590-CGW with the current BIOS and memory kit(s). Perhaps a newer BIOS will help with compatibility down the line, as we’ve seen at least one review reach DDR4 3600 speeds. For now, stick with DDR4 3200 and the tightest timings you can afford to buy. We’d also like to see the price lowered to match the feature set and appearance to compete better in the market.

As far as comparable motherboards go, there are several around this price point, including the MSI Z590 Unify ($379.99), ASRock Z590 Taichi (now $429), Asus ROG Strix Z590-E Gaming Wifi ($379.99) and the Gigabyte Z590 Aorus Master (now $409.99). The most noticeable difference between these boards is appearance. Where all of the aforementioned competition has a more premium look, including integrated RGB lighting and more shrouds covering the board, the SuperO isn’t in the same league. All boards have three (or more) M.2 sockets. Others include six SATA ports (though there is likely lane sharing).

If you’re into a fast, wired network, the SuperO is the board to own with its 10GbE, but Wi-Fi goes to the others that all have 6E hardware. Last but not least, memory support on the competition is currently better. If you plan to run anything past DDR4 3200, grab a different board to save yourself some trouble.

In the end, the SuperO C9Z590-CGW is an interesting entity in the mid-range market. It’s supposed to be a gaming motherboard, but I feel the server DNA gets in the way. Out of the box, it acts similar to a server board, following Intel specifications, but can run without those limits and perform on par with other boards. If you’re looking for the least-expensive Z590-based motherboard with a 10GbE, you’ve found it. But if that isn’t a big deal for you, better-looking, better-equipped and lower-priced boards are available.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Thanks to Anandtech, we have information on two new Supermicro motherboards designed for Intel's new Ice Lake Xeon Scalable CPUs: the 12DPL-NT6 and X12DPL-i6. The boards feature dual LGA4189 sockets -- for a max potential configuration of 80 CPU cores all squished into a standard ATX form factor.

The stand-out feature of the X12DPL-NT6 and X12DPL-i6 is the ATX form factor; Supermicro has demonstrated it can put two gigantic LGA4189 sockets onto an ATX board without sacrificing many features; the two sockets alone take up nearly half of the entire board's size.

The only feature Supermicro had cut out was the platform's maximum support of twelve DIMM slots. Due to the size constraints, each CPU can only access up to four DIMM slots (eight total on the board), meaning each chip will be limited to quad-channel memory configurations. This will only be a problem if you're workloads benefit from significantly high memory bandwidth/capacity.

For connectivity, you get four PCIe 4.0 slots on the bottom, with each slot having the full 16 lanes available. For storage, the boards support twelve SATA slots with RAID 0, 1, 5, and 10 array support, and dual M.2 PCIe Gen 4.0 slots.

The only difference between the X12DPL-NT6 and X12DPL-i6 is the ethernet and M.2 configuration. With the X12DPL-NT6 you get dual Intel X550 10Gb Ethernet ports, and one more M.2 slot capable of PCIe Gen 4.0 x8 support.

The lower end X12DPL-i6 does not include 10Gb Ethernet and instead relies on two Intel i210 Gigabit LAN controllers for network connectivity. It also loses the x8 M.2 slot.

We have no idea of when these boards will be available to purchase, but if you're in the market for something like this, Supermicro will probably happily return your emails or phone calls.

Supermicro's 1023US-TR4 is a slim 1U dual-socket server designed for high-density compute environments in high-end cloud computing, virtualization, and enterprise applications. With support for AMD's EPYC 7001 and 7002 processors, this high-end server packs up to two 64-core Eypc Rome processors, allowing it to cram 128 cores and 256 threads into one slim chassis. 

We're on the cusp of Intel's Ice Lake and AMD's EPYC Milan launches, which promise to reignite the fierce competition between the long-time x86 rivals. In preparation for the new launches, we've been working on a new set of benchmarks for our server testing, and that's given us a pretty good look at the state of the server market as it stands today. 

We used the Supermicro 1023US-TR4 server for EPYC Rome testing, and we'll focus on examining the platform in this article. Naturally, we'll add in Ice Lake and EPYC Milan testing as soon as those chips are available. In the meantime, here's a look at some of our new benchmarks and the current state of the data center CPU performance hierarchy in several hotly-contested price ranges. 

Inside the Supermicro 1023US-TR4 Server

The Supermicro 1023US-TR4 server comes in the slim 1U form factor. And despite its slim stature, it can host an incredible amount of compute horsepower under the hood. The server supports AMD's EPYC 7001 and 7002 series chips, with the latter series topping out at 64 cores apiece, which translates to 128 cores and 256 threads spread across the dual sockets.

Support for the 7002 series chips requires a 2.x board revision, and the server can accommodate CPU cTDP's up to 280W. That means it can accommodate the beefiest of EPYC chips, which currently comes in the form of the 280W 64-core EPYC 7H12 with a 280W TDP. 

The server has a tool-less rail mounting system that eases installation into server racks and the CSE-819UTS-R1K02P-T chassis measures 1.7 x 17.2 x 29 inches, ensuring broad compatibility with standard 19-inch server racks. 

The front panel comes with standard indicator lights, like a unit identification (UID) light that helps with locating the server in a rack, along with drive activity, power, status light (to indicate fan failures or system overheating), and two LAN activity LEDs. Power and reset buttons are also present at the upper right of the front panel.

By default, the system comes with four tool-less 3.5-inch hot-swap SATA 3 drive bays, but you can configure the server to accept four NVMe drives on the front panel, and an additional two M.2 drives internally. You can also add an optional SAS card to enable support for SAS storage devices. The front of the system also houses a slide-out service/asset tag identifier card to the upper left. 

Popping the top off the chassis reveals two shrouds that direct air from the two rows of hot-swappable fans. A total of eight fan housings feed air to the system, and each housing includes two counter-rotating 4cm fans for maximum static pressure and reduced vibration. As expected with servers intended for 24/7 operation, the system can continue to function in the event of a fan failure. However, the remainder of the fans will automatically run at full speed if the system detects a failure. Naturally, these fans are loud, but that's not a concern for a server environment.  

Two fan housings are assigned to cool each CPU, and a simple black plastic shroud directs air to the heatsinks underneath. Dual SP3 sockets house both processors, and they're covered by standard heatsinks that are optimized for linear airflow. 

A total of 16 memory slots flank each processor, for a total of 32 memory slots that support up to 4TB of registered ECC DDR4-2666 with EPYC 7001 processors, or an incredible 8TB of ECC DDR4-3200 memory (via 256GB DIMMs) with the 7002 models, easily outstripping the memory capacity available with competing Intel platforms.

We tested the EPYC processors with 16x 32GB DDR4-3200 Samsung modules for a total memory capacity of 512GB. In contrast, we loaded down the Xeon comparison platform with 12x 32GB Sk hynix DDR4-2933 modules, for a total capacity of 384GB of memory. 

The H11DSU-iN motherboard's expansion slots consist of two full-height 9.5-inch PCIe 3.0 slots and one low-profile PCIe 3.0 x8 slot, all mounted on riser cards. An additional internal PCIe 3.0 x8 slot is also available, but this slot only accepts proprietary Supermicro RAID cards. All told, the system exposes a total of 64 lanes (16 via NVMe storage devices) to the user. 

As one would imagine, Supermicro has other server offerings that expose more of EPYCs available 128 lanes to the user and also come with the faster PCIe 4.0 interface. 

The rear I/O panel includes four gigabit RJ45 LAN ports powered by an Intel i350-AM4 controller, along with a dedicated IPMI port for management. Here we find the only USB ports on the machine, which come in the form of two USB 3.0 headers, along with a COM and VGA port. 

Two 1000W Titanium-Level (96%+) redundant power supplies provide power to the server, with automatic failover in the event of a failure, as well as hot-swapability for easy servicing. 

The BIOS is easy to access and use, while the IPMI web interface provides a wealth of monitoring capabilities and easy remote management that matches the type of functionality available with Xeon platforms. Among many options, you can update the BIOS, use the KVM-over-LAN remote console, monitor power consumption, access health event logs, monitor and adjust fan speeds, and monitor the CPU, DIMM, and chipset temperatures and voltages. Supermicro's remote management suite is polished and easy to use, which stands in contrast to other platforms we've tested. 

Test Setup

Here we can see the selection of processors we've tested for this review, though we use the Xeon Platinum Gold 8280 as a stand-in for the less expensive Xeon Gold 6258R. These two chips are identical and provide the same level of performance, with the difference boiling down to the more expensive 8280 coming with support for quad-socket servers, while the Xeon Gold 6258R tops out at dual-socket support. 

To assess performance with a range of different potential configurations, we used the Supermicro 1024US-TR4 server with four different EPYC Rome configurations. We outfitted this server with 16x 32GB Samsung ECC DDR4-3200 memory modules, ensuring that both chips had all eight memory channels populated. 

We used a Dell/EMC PowerEdge R460 server to test the Xeon processors in our test group, giving us a good sense of performance with competing Intel systems. We equipped this server with 12x 32GB Sk hynix DDR4-2933 modules, again ensuring that each Xeon chip's six memory channels were populated. These configurations give the AMD-powered platform a memory capacity advantage, but come as an unavoidable side effect of the capabilities of each platform. As such, bear in mind that memory capacity disparities may impact the results below.  

We used the Phoronix Test Suite for testing. This automated test suite simplifies running complex benchmarks in the Linux environment. The test suite is maintained by Phoronix, and it installs all needed dependencies and the test library includes 450 benchmarks and 100 test suites (and counting). Phoronix also maintains openbenchmarking.org, which is an online repository for uploading test results into a centralized database. We used Ubuntu 20.04 LTS and the default Phoronix test configurations with the GCC compiler for all tests below. We also tested both platforms with all available security mitigations. 

Linux Kernel and LLVM Compilation Benchmarks

We used the 1023US-TR4 for testing with all of the EPYC processors in the chart, and here we see the expected scaling in the timed Linux kernel compile test with the AMD EPYC processors taking the lead over the Xeon chips at any given core count. The dual EPYC 7742 processors complete the benchmark, which builds the Linux kernel at default settings, in 21 seconds. The dual 24-core EPYC 7F72 configuration is impressive in its own right — it chewed through the test in 25 seconds, edging past the dual-processor Xeon 8280 platform. 

AMD's EPYC delivers even stronger performance in the timed LLVM compilation benchmark — the dual 16-core 7F72's even beat the dual 28-core 8280's. Performance scaling is somewhat muted between the flagship 64-core 7742 and the 24-core 7F72, largely due to the strength of the latter's much higher base and boost frequencies. That impressive performance comes at the cost of a 240W TDP rating, but the Supermicro server handles the increased thermal output easily. 

Molecular Dynamics and Parallel Compute Benchmarks

NAMD is a parallel molecular dynamics code designed to scale well with additional compute resources; it scales up to 500,000 cores and is one of the premier benchmarks used to quantify performance with simulation code. The EPYC processors are obviously well-suited for these types of highly-parallelized workloads due to their prodigious core counts, with the dual 7742 configuration completing the workload 28% faster than the dual Xeon 8280 setup. 

Stockfish is a chess engine designed for the utmost in scalability across increased core counts — it can scale up to 512 threads. Here we can see that this massively parallel code scales well with EPYC's leading core counts. But, as evidenced by the dual 24-core 7F72's effectively tying the 28-core Xeon 8280's, the benchmark also generally responds well to the EPYC processors. The dual 16-core 7F52 configuration also beat out both of the 16-core Intel comparables. Intel does pull off a win as the eight-core 6250 processors beat the 7F32's, though.  

We see similarly impressive performance in other molecular dynamics workloads, like the Gromacs water benchmark that simulates Newtonian equations of motion with hundreds of millions of particles and the NAS Parallel Benchmarks (NPB) suite. NPB characterizes Computational Fluid Dynamics (CFD) applications, and NASA designed it to measure performance from smaller CFD applications up to "embarrassingly parallel" operations. The BT.C test measures Block Tri-Diagonal solver performance, while the LU.C test measures performance with a lower-upper Gauss-Seidel solver. 

Regardless of the workload, the EPYC processors deliver a brutal level of performance in highly-parallelized applications, and the Supermicro server handled the heat output without issue. 

Rendering Benchmarks

Turning to more standard fare, provided you can keep the cores fed with data, most modern rendering applications also take full advantage of the compute resources. Given the well-known strengths of EPYC's core-heavy approach, it isn't surprising to see the 64-core EPYC 7742 processors carve out a commanding lead in the C-Ray and Blender benchmarks. Still, it is impressive to see the 7Fx2 models beat the competing Xeon processors with similar core counts nearly across the board. 

The performance picture changes somewhat with the Embree benchmarks, which test high-performance ray tracing libraries developed at Intel Labs. Naturally, the Xeon processors take the lead in the Asian Dragon renders, but the crown renders show that AMD's EPYC can offer leading performance even with code that is heavily optimized for Xeon processors. 

Encoding Benchmarks

Encoders tend to present a different type of challenge: As we can see with the VP9 libvpx benchmark, they often don't scale well with increased core counts. Instead, they often benefit from per-core performance and other factors, like cache capacity. 

However, newer encoders, like Intel's SVT-AV1, are designed to leverage multi-threading more fully to extract faster performance for live encoding/transcoding video applications. Again, we can see the impact of EPYC's increased core counts paired with its strong per-core performance as the EPYC 7742 and 7F72 post impressive wins. 

Python and Sysbench Benchmarks

The Pybench and Numpy benchmarks are used as a general litmus test of Python performance, and as we can see, these tests don't scale well with increased core counts. That allows the Xeon 6250, which has the highest boost frequency of the test pool at 4.5 GHz, to take the lead. 

Compression and Security

Compression workloads also come in many flavors. The 7-Zip (p7zip) benchmark exposes the heights of theoretical compression performance because it runs directly from main memory, allowing both memory throughput and core counts to impact performance heavily. As we can see, this benefits the EPYC 7742 tremendously, but it is noteworthy that the 28-core Xeon 8280 offers far more performance than the 24-core 7F72 if we normalize throughput based on core counts. In contrast, the gzip benchmark, which compresses two copies of the Linux 4.13 kernel source tree, responds well to speedy clock rates, giving the eight-core Xeon 6250 the lead due to its 4.5 GHz boost clock. 

The open-source OpenSSL toolkit uses SSL and TLS protocols to measure RSA 4096-bit performance. As we can see, this test favors the EPYC processors due to its parallelized nature, but offloading this type of workload to dedicated accelerators is becoming more common for environments with heavy requirements. 

SPEC CPU 2017 Estimated Scores

We used the GCC compiler and the default Phoronix test settings for these SPEC CPU 2017 test results. SPEC results are highly contested and can be impacted heavily with various compilers and flags, so we're sticking with a bog-standard configuration to provide as level of a playing field as possible. It's noteworthy that these results haven't been submitted to the SPEC committee for verification, so they aren't official. Instead, view the above tests as estimates, based on our testing.

The Xeon 6250 and 8280 processors take the lead in the single-threaded intrate tests, while the AMD EPYC processors post impressively-strong single-core measurements in the fprate tests. 

Conclusion

AMD has enjoyed a slow but steadily-increasing portion of the data center market, and much of its continued growth hinges on increasing adoption beyond hyperscale cloud providers to more standard enterprise applications. That requires a dual-pronged approach of not only offering a tangible performance advantage, particularly in workloads that are sensitive to per-core performance, but also having an ecosystem of fully-validated OEM platforms readily available on the market. 

The Supermicro 1023US-TR4 server slots into AMD's expanding constellation of OEM EPYC systems and also allows discerning customers to upgrade from the standard 7002 series processors to the high-frequency H- and F-series models as well. It also supports up to 8TB of ECC memory, which is an incredible amount of available capacity for memory-intensive workloads. Notably, the system comes with the PCIe 3.0 interface while the second-gen EPYC processors support PCIe 4.0, but this arrangement allows customers that don't plan to use PCIe 4.0 devices to procure systems at a lower price point. As one would imagine, Supermicro has other offerings that support the faster interface. 

Overall we found the platform to be robust, and out-of-the-box installation was simple with a tool-less rail kit and an easily-accessible IPMI interface that offers a cornucopia of management and monitoring capabilities. Our only minor complaints are that the front panel could use a few USB ports for easier physical connectivity. The addition of a faster embedded networking interface would also free up an additional PCIe slot. Naturally, higher-end Supermicro platforms come with these features. 

As seen throughout our testing, the Supermicro 1023US-TR4 server performed admirably and didn't suffer from any thermal throttling issues regardless of the EPYC processors we used, which is an important consideration. Overall, the Supermicro 1023US-TR4 server packs quite the punch in a small form factor that enables incredibly powerful and dense compute deployments in cloud, virtualization, and enterprise applications. 

MORE: Best CPUs

MORE: CPU Benchmarks and Hierarchy

MORE: All CPUs Content

We recently reported about Expanscape, a startup developing battlestation laptops featuring up to seven displays and offering their prototypes to interested parties at undisclosed prices. Today we can share some more information about pricing, which tops out at an eye-popping $20,692 (after conversion) for the seven-screen model's base configuration that's powered by an AMD Ryzen 5950X processor paired with Nvidia RTX 3070 (8GB) GPUs.

Expanscape's Aurora laptops with five or seven screens are a work in progress, so every unit is unique to a large degree. The manufacturer says that it is getting closer to finalized pricing for its A7 prototype as it had standardized on specific hardware, but until now, the company hasn't announced official prices of its multi-monitor laptops. 

Expanscape currently has two types of laptop prototypes in three configurations. The 'basic' Aurora A5 notebook comes with five monitors: four 15.6-inch 4K displays and one 7-inch touchscreen integrated into its palm rest. This system packs an eight-core AMD Ryzen 7 4800U processor paired with 64GB of DDR4 memory, a 2TB PCIe/NVMe SSD, and a 2TB SATA SSD. This entry-level machine currently costs approximately £4,500, or $6,286 USD after conversion. 

The 'full' Aurora A7 laptops are equipped with seven displays and come in two configurations powered by an AMD Ryzen 5950X processor paired with Nvidia RTX 3070 (8GB) GPUs. One model features four 17.3-inch 4K monitors, two 9.7-inch 1536p monitors, and one 7-inch touchscreen. Another does not have the 7-inch touchscreen but comes with a 17.3-inch touch-enabled LCD in its base to replace the keyboard, which now extends from under the chassis. 

Expanscape's Aurora A7 notebooks with seven screens are naturally more expensive than their smaller A5 brethren. At present, these machines cost £15,000 ($20,952 USD after conversion) for a standard model, but the pricing goes up with all of the customizations required for the professional built-to-order systems. All Aurora A7 machines come equipped with 128GB of DDR4 memory as well as 8TB of PCIe/NVMe and SATA storage. 

Both the A7 and A5 models come with internal 8 megapixel cameras with triple LED lighting and have dedicated hardware switches that can disable the cameras, microphone, speakers and wireless features. 

Notably, the current pricing of Expanscape's Aurora machines is somewhat higher than several months ago because of component shortages and new policies at the UK border. 

It is noteworthy that while Expanscape's Aurora notebooks are still prototypes, they are rather clumsy and heavy. However, according to the manufacturer, demand for these systems is still fairly significant. Customers who bought the systems reportedly said that they needed them 'yesterday.' 

What remains to be seen is whether high demand for Expanscape's Aurora A5 and Aurora A7 will enable the company to make them look and feel like commercial products and not just prototypes. Evidently, a more solid build will make these systems considerably more popular among interested customers. 

What if you need a mobile system with more than three screens? Well, Expanscape has developed a prototype of a laptop with as many as seven screens, and it is already selling the prototypes to interested customers. It also comes packed with an impressive amount of power on the compute side to match. 

The Aurora 7 Prototype indeed comes with four 17.3-inch monitors featuring a 4K resolution (two working in landscape, two in portrait mode) as well as three auxiliary 7-inch screens featuring a 1920 x 1200 resolution. All the monitors fold or swivel out of the primary chassis, making it a transformer of sorts, and no on-site assembly before deployment is necessary. The whole system weighs around 12 kilograms, so it is not easy to carry, but it is naturally easier to transport than a laptop along with six extra displays. 

With more of us working from home due to the pandemic, multi-display setups are becoming the norm and are widely used for a variety of applications. Setting a multi-monitor configuration at home or in an office is easy, and while attaching two more displays to a laptop is also possible, it gets slightly more complicated.

Expanscape's Aurora 7 Prototype laptop computer is built for very particular applications and audiences in mind (such as security operations centers, data scientists, content creators) that traditionally use multi-display PCs, but who at times need to transport and deploy them quickly. The creators wanted their seven-screen laptop computer to be portable, structurally rigid, and capable of running demanding programs. 

As far as internal hardware is concerned, the Aurora 7 is powered by Intel's Core i9-9900K processor that is accompanied by 64GB of DDR4-2666 memory, Nvidia's GeForce GTX 1060 graphics card, two PCIe 3.0 x4 M.2 SSDs, one 2.5-inch MLC SSD, and a 2TB 7200RPM hard drive. The PC has all modern connectivity technologies, including Bluetooth, GbE, Wi-Fi, and USB. Since the Aurora 7 uses a fairly spacious chassis, the developer says that it can use different platforms, including AMD's Ryzen 9 3950X or Intel's Core i9-10900K. 

Among the impressive peculiarities of the Expanscape's Aurora 7 Prototype are two internal batteries. The primary internal battery features an 82Wh capacity and powers the system itself. The secondary internal battery has a 148Wh capacity and is used to power the screens. The battery life for the whole system is about 2 hours 20 minutes, but at high clocks under high loads, it will get lower. 

Technically, all of Expanscape's seven-screen Aurora 7 machines are just prototypes that do not look or feel like commercial products, yet the company can build them to order and sell to interested parties who agree so sign a contract and pay a hefty sum of money. 

Supermicro has announced its first workstation based on AMD's Ryzen Threadripper Pro processor. The new system can pack a CPU with up to 64 cores, up to 2TB of RAM, and up to four double-wide graphics cards or accelerators. The machine is designed for AI/DL, digital content creation, and engineering simulations.  

Historically, AMD has been quite keen on winning designs with server makers, but when it comes to workstations from professional suppliers, the company has never been truly profound. Perhaps because high-end/corporate workstations is not exactly a high-volume business and spending resources on winning servers is always more fruitful. Yet, with its Ryzen Threadripper Pro 3000WX-series, AMD clearly wants to enter this market. The Supermicro A+ SuperWorkstation 5014A-TT is the industry's second workstation based on AMD's Ryzen Threadripper Pro 3000WX-series processor after Lenovo's ThinkStation P620 machine and it actually has an edge over its only rival as it supports more dual-slot PCIe 4.0 x16 cards, according to its manufacturer. 

The A+ SuperWorkstation 5014A-TT workstation is based on the AMD WRX80 platform with all of its advantages, such as an eight-channel memory subsystem that supports up to 2TB of DDR4-3200 SDRAM with or without ECC as well as 128 usable PCIe 4.0 lanes. The M12SWA-TF motherboard that powers the workstation carries six PCIe 4.0 x16 slots, four M.2-2280/22110 slots for SSDs, two U.2 sockets, one GbE port (Intel I210AT), one 10GbE connector (Marvell AQC113C), an ASpeed AST2600 baseboard management controller, a TPM 2.0 header, multiple USB 3.2 Gen 2 Type-A & Type-C ports, and a 7.1-audio codec.

The machine itself measures 535×222×573 mm and is equipped with an up to 2000W 80Plus Platinum-certified PSU, so it will handle virtually all combinations of AMD's Ryzen Threadripper Pro CPUs with any graphics cards, compute accelerators, SSDs, and memory modules available today. Meanwhile, since the A+ SuperWorkstation 5014A-TT is listed as 'coming soon', Supermicro does not disclose which add-in cards and storage devices it will offer with the system. 

The manufacturer has not yet disclosed pricing of its A+ SuperWorkstation 5014A-TT machines, but it is logical to expect its MSRP to be comparable to that of Lenovo's ThinkStation P620.

Supermicro is best known to most for server hardware. But over the last few years, the company has dipped its toes in the gaming realm. We’ve seen X99 and X299, as well as Z170 through Z390-based motherboards from the company, culminating with the C9Z490-PGW we have for review today. Like past iterations, the C9Z490-PGW board contains server-level features including PXE boot, four PCIe slots served by a PLX chip, in addition to parts and features we expect on a gaming motherboard.

With an MSRP of $395, this places the C9Z490-PGW squarely in the crosshairs of mid/upper mid-range Z490-based motherboards. Our PGW comes equipped with the full complement of six SATA ports, two M.2 sockets, Aquantia 10 GbE and Wi-Fi 6, a premium Realtek audio solution and more. There’s also the four PCIe slots that use a PLX chip for additional lanes. These are useful for those using multiple video cards, especially with compute workflows.

Performance on this board was on par with any board that follows intel specifications out of the box. To that end, other motherboards (that go beyond the spec) performed better in extended tests. However, you’ll see that the results weren't that different overall. If you remove the stock turbo limits, then it is no different. Gaming performance was also on par with the other boards. The C9Z490-PGW happily handled our Intel Core i9-10900K CPU at 5.2 GHz without issue. RAM overclocking wasn’t quite set-it-and-go at DDR4 4000, but with a few tweaks, we ran our kit without issue.

Other features on the C9Z490-PGW include onboard power/reset buttons, five 4-pin fan headers, four SATA ports and two USB 3.2 Gen2 Type-C ports (one Gen2x2/20 Gbps). For those who would like to add RGB lighting to this board, you’re out of luck. There are no onboard headers for lighting, nor any integrated RGB. 

Below is a complete list of specifications from Supermicro. Getting past that, we’ll tour around the board in detail.

Specifications - Supermicro C9Z490-PGW

Starting with accessories, Supermicro includes what’s needed to get started, but not much else. Below is a list of what is included in the box along with the board.

• Quick reference guide
• Driver disk
• (4) SATA cables
• Labels for cables
• IO plate
• Wi-Fi antenna

Compared to most modern boards in this price range, the Supermicro C9Z940-PGW is far from flashy. A simple matte-black PCB sporting black heatsinks on all of the toasty parts. Contrasting this barrage of black are the silver/metal-colored reinforced DRAM and PCIe slots. The board’s appearance isn’t exactly offputting, but at this price point there are better-looking boards available, especially if shrouds and RGB lighting are your thing.

Speaking of RGB, the C9Z490-PGW doesn’t have any integrated on the board. Instead, it has a single white LED illuminating the SuperO name and a small strip above the rear IO. In fact, the board doesn’t have any RGB headers at all. Any RGB lighting will have to come from third-party ecosystems and controllers. That said, if you’re looking for a board like this in the first place, chances are you won’t miss the lack of lighting and basic styling.

Looking closely at the top half of the board, we’ll start out on the left side. Here we see the black plastic The C9Z490-PGW uses well-apportioned heatsinks covering the VRMs, including other hot-running chips (like the PLX chip just above the top PCIe slot, for example).

Feeding power to the CPU is an 8-pin EPS plug located between the VRM heatsinks. The first of five 4-pin fan headers are located here, just next to the EPS connector. The second header is found just to the right of the DRAM slots. All fan headers support both PWM and DC control fans up to 1A/12W (the manual doesn’t list any power, so we assume the standard).

Next up are the four reinforced DRAM slots that alternate in grey and black (grey slots are the first to be populated). The locking mechanism is single-sided, which is helpful considering the proximity of the top PCIe slot. To the right of these are two more fan headers along with power and reset buttons. Sliding further down the right edge, we see the 24-pin ATX connector along with a front-panel USB 3.2 Gen2 Type-C header.

The C9Z490-PPGW and its Wi-Fi free brother (C9Z490-PG, the W stands for Wi-Fi) come equipped with an 8+2 VRM setup. Power goes through an Infineon XDPE12284C controller (X+Y=8) that feeds six 90A Infineon TDA21490 MOSFETs for Vcore, along with 70A Infineon TDA21535 MOSFETs for the SOC. The phase count may not be as high as others. However, the premium Infineon ICs do a great job at handling our power-hungry Intel Core i9-10900K, even when overclocked to 5.2 GHz.

Moving down to the bottom half of the board, we’ll look at the audio, PCIe slot breakdown and SATA storage. Starting on the left side, we see the Realtek ALC1220 IC, along with a few small audio capacitors. Just above this, under a black heatsink, is the Aquantia AQC107 10 GbE chip. There is some separation of audio bits to the rest of the board. But otherwise, there aren’t any covers on the IC to protect against EMI. That said, this implementation using the premium audio codec should be good enough for most users.

In the middle of the board, we see four full-length PCIe slots, each using reinforcement to prevent shearing and protect against EMI. The four slots work in a x16/x0/x16/x0 or x8/x8/x8/x8 configuration due to the PEX 8747 PCIe Gen3 switch that adds lanes (at the expense of some latency) to the CPU. This configuration is suitable for multi-GPU setups -- in particular, compute type loads allowing users to stuff the slots full of cards. For gaming, Supermicro lists AMD CrossfireX as supported, but no mention of SLI.  In addition to the four full-length slots is a single x1 size slot that is fed from the chipset.

Wedged between PCIe slots 1/2 and 3/4, are two M.2 sockets. The top socket supports drives up to 80mm, while the bottom up to 110mm. The top slot supports a maximum of PCIe 3.0 x4 speeds while the bottom slot is up to PCIe 3.0 x2. Both sockets only support PCIe-based modules as well as support RAID0 and 1. I would like to have seen at least one of these support SATA-based devices, especially considering there are only four SATA ports available.

As just mentioned, the C9Z490-PGW includes four SATA ports that support RAID0, 1, 5 and 10. All ports should remain enabled regardless of how the M.2 sockets are populated. For this type of board, I would like to have seen all six ports available, giving users more choices in saturating their storage options. In this same area is a 2-character debug LED that’s useful during POST to determine where the system may be hanging.

The bottom edge has a slew of headers including front panel audio, USB and more. In addition to the debug LED above, it also sports a 4 LED debug that stays lit if there is an issue during POST (CPU, DIMM, VGA and BOOT). Along with the debug LED, these can help pinpoint issues that occur before reaching Windows. Below is a list of all headers ordered from left to right.

• Front panel audio
• SMB to PCIe slots
•  USB 2.0 header
•  Standby power header
• USB 3.2 Gen1 header
• TPM header
• Debug LEDs
• USB 2.0 header
• COM port
• Front panel header

The rear IO is full of inputs and outputs including Wi-Fi, dual Ethernet, audio, USB and video ports. On the video side is a DisplayPort and HDMI port for use with CPUs that have integrated graphics. There are a total of six USB ports out back - two USB 3.2 Gen1 (blue) and four USB 3.2 Gen 2 (one Type-C is Gen2x2, 20 Gbps). Our only concern here is the USB port count sits at six, which may not be enough for some users.

Above the USB ports are the Intel I219-V (left) and the Aquantia 10 GbE LAN ports. The C9Z490-PGW is the least -expensive motherboard that includes a 10 GbE port. The audio stack consists of five analog plugs and SPDIF. Last but not least is the small CMOS reset button.

Software

On the software front, Supermicro uses its SuperOBooster application, an all-in-one monitoring and adjustment tool designed for its motherboards. The software is easy to read and maneuver around and controls the CPU and memory, fans, voltage, and updating the BIOS. SuperOBooster does a good job of presenting system information as well as being simple to use.

Firmware

To give you a sense of the Firmware, we’ve gathered screenshots showing most BIOS screens.

The BIOS on these Supermicro boards do the job and have improved since they first came out. however, they aren’t as refined as the others. It feels like a server BIOS in that the look is fairly simple. In EZ Mode, you’re able to get an idea of where the system stands and change a couple of options. Using the Advanced mode, you’ll see headers across the top (Main, Overclocking, Advanced, H/W Monitor, Boot, Save & Exit and BIOS Update) with options listed below.

Although the BIOS works just fine, POST takes a lot longer than most consumer-facing motherboards. The way your selection is highlighted works differently so it takes a bit of getting used to. In essence, you have two things highlighted at once and have to hit enter to select the field to get it to move. Typically, the highlight follows the cursor without making a selection.

Test System

Our test system uses Windows 10 64-bit OS (1909) with all threat mitigations applied. The motherboard BIOS used is the latest non-beta available to the public, unless otherwise noted. The hardware used is as follows:

Benchmark Settings

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content 

All standard benchmarks and power tests are performed using the CPU’s stock frequencies (including stock Thermal Velocity Boost), with all power-saving features enabled. Optimized defaults are set in the BIOS and the memory is set using the XMP profiles. For this baseline testing, Windows is set to High Performance, except for during power testing, where we switch to Balanced so the PC idles properly.

**The red bar represents the board we are reviewing, not the fastest/slowest result for a given chart.

Synthetic Benchmarks

Synthetics are a great tool to figure out if a board runs out of spec, as identical settings should produce extremely similar performance results. Advanced memory timings are the one place where motherboard makers can still optimize for either stability or performance though, and those settings can impact some testing.

Results in our synthetic benchmarks matched other boards in both single-threaded and multi-threaded applications for the most part. Our 7Zip results are lower than most, due to the board following intel specifications. We’ll see this theme a couple of times in our testing. The MS Office portion of our testing showed some inconsistencies and was slightly slower than the other boards tested. In short, if you want to get the most out of this system, you’ll want to raise the intel power limits for prolonged turbo boost and sustained clock speeds.

Timed Applications

During the timed applications, the C9Z490-PGW was slower than most boards in both iterations of Handbrake (x264/x265). LAME results were close to other boards but still on the slower side while the Corona benchmark result was a lot slower, again running into intel parameters slowing things down.

3D Games and 3DMark

We’ve recently updated our game tests to The Division 2 and Forza Horizon 4. The games were run at 1920x1080 resolution using the Ultra preset. As the resolution goes up, the CPU tends to have less of an impact on most games. The goal with these settings is to determine if there are differences in performance at the most commonly used resolution with settings most people use or at least strive for.

On the gaming front, our board did well overall. 3DMark results were a bit slower, but games didn’t show much slower frame rates with the system’s performance within the margin of error here.

Power Consumption / VRM Temperatures

For power testing with our current-hungry Core i9 CPU, we used AIDA64’s System Stability Test with Stress CPU, FPU and Cache enabled, using the peak power consumption value within the first minute. The wattage reading is from the wall via a Kill-A-Watt meter to capture the entire ecosystem (minus the monitor). The only variable that changes in these reviews is the motherboard; all other parts are the same.

Our Supermicro board’s idle power use sat at 61W on the desktop, using the balanced power profile. This value is one of the higher end of what we’ve seen, with most boards sitting around the upper 40s to low 50s. Load wattage was also on the higher side, at 291W peak during the AIDA64 stress test. This averages out to 176W, which is the most of any Z490 based board we’ve tested. Still, the difference is only a handful of watts and won’t be noticed on your power bill.

After taking the shackles off and overclocking our CPU to 5.2 GHz around 1.35V, temperatures went up as expected, peaking at 73 degrees Celsius. This puts it on par with a few other boards we tested, including the EVGA Z490 FTW.

Looking at the VRM temperatures, during our stock run, the board peaked at just under 50 degrees Celsius. This is a good result and better than most boards. But again, this board does follow Intel’s specifications for Turbo out of the box. In the chart, you can see where the CPU temperature drops dramatically as it comes down off of the turbo clock speeds and voltages, thus lowering VRM temperatures.

Overclocking

As with most of our previous Z490 reviews, the Supermicro C9Z490-PGW ran our Core i9-10900K CPU at 5.2 GHz and about 1.35V while keeping the CPU around 90 degrees Celsius. The 90A MOSFETs handled the CPU without issue, though they will get warm during heavy testing/use. We experienced some vdroop, but this was easily managed by adjusting Load-Line Calibration (LLC) and setting it to Level 4.

On the memory side of things, we installed our DDR4 4000 kit, set XMP and it didn’t want to boot. After making some manual changes (setting VccSA and VccIO voltages) and leaving XMP set, the system booted without issue. DDR4 4000 is the limit of this board, so I’m not surprised some tweaking was needed to get there. You won’t break any memory speed records on this board, but it’s not designed for such things in the first place. The C9Z490-PGW is intended for stability and reliability, not breakneck RAM records.

Final Thoughts

Wrapping things up, the Supermicro C9Z490-PGW performed fine in our test suite. We don’t say great because it’s hindered by the adherence to the Intel specifications in some tests. If you take the chains off and let it rip (as many motherboards do by default), performance would be on par with the other tested boards.

On the features side, the board brings robust power delivery with high-quality 90A MOSFETs, an Aquantia 10 GbE and Wi-Fi 6, a PLX chip for additional PCIe lanes, and even the ability to PXE boot (boot from a network card). Additionally, the board includes a premium audio codec, supports CrossfireX for multiple GPUs, and an ultra-fast USB 3.2 Gen2x2 Type-C port. Overall the C9Z490-PGW is a very competent board, though not without its quirks.

The Supermicro BIOS has improved over the years, but still doesn’t feel quite as refined as others. The highlighting and selecting are different than other boards, but you get used to it. Boot times out of the box are also a lot longer and need some tweaking (disabling unused items) to lower times. Also, there are better-looking boards available for less money, though none that include 10 GbE or a PLX chip for that matter.

Around its current $342.99 price at Newegg, this board competes with the likes of the ASRock Z490 Taichi ($369.99), Asus ROG Maximus XII Hero ($393.99), Gigabyte Z490 Aorus Master ($329.99), and the MSI MEG Z490 Ace ($399.99). All boards offer a solid foundation for building a Z490-based PC from, look good, and perform well. What they are missing however is the additional features of the Supermicro, including the 10 GbE, four full-length x8 capable PCIe slots, and more.

Supermicro continues to improve on its foray into gaming motherboards. The C9Z490-PGW offers users a solid crossover option between a gaming board and server/workstation-class boards. You get some of the benefits of using a device that is geared towards that environment while still retaining the ability to overclock on reliable, robust hardware. If you don’t need these features and looks are important, other options may be a better fit. If you need these features, at $342, it is somewhat of a steal, especially if you are overclocking and can make use of the 10GbE port, while not planning to showcase the board for its looks.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content 

If you haven't heard of the gaming brand SuperO, we wouldn't blame you. Supermicro's gaming division isn't very well-known in the Western market. Nevertheless, SuperO's latest motherboard is pretty interesting. The Supermicro SuperO 9Z490-PGW is a high-end board with a total of four PCIe 3.0 slots, as spotted by Hermitage Akihabara. It is built on the new Z490 chipset with an LGA1200 socket, and, thus, for use with Intel's upcoming Comet Lake-S CPUs.

Typically, the Z490 platform doesn't support this many PCIe 3.0 slots. There are some boards that have this many slots, but few of them actually have all 16 lanes accessible on each slot. Supermicro's trick is to use a PLX lane switch chip, which essentially splits the PCIe lanes from the processor into multiple lanes. This overcomes lane restrictions put in place by graphics cards. Bandwidth per card is still limited.

Next to that unique feature, the board comes with all the standard goodies you would expect on a Z490 motherboard, including Intel i219V Gigabit Ethernet, Bluetooth 5.0, 802.11ax WiFi, four DDR4 RAM slots, dual M.2 slots and "enterprise quality" VRM circuitry. Iteven comes with 10 Gigabit AQC107 Ethernet built by Aquantia. A Realtek ALC1220 chip on a dedicated portion of the board handles audio. 

Intriguingly, despite its high-end positioning, it appears that the SuperMicro C9Z490-PGW doesn't come with RGB lighting. This paired with the PLX chip tells us that despite being labeled as a "gaming" motherboard, the new SuperO Z490 board is fit for a professional audience too. 

There's no word on pricing yet. However, PLX chips have become quite expensive in recent years, so don't expect this board to come cheap.

The era of 5G is coming, and it has the potential to shake up mobile internet connections in a big way. It can enable much faster data speeds for simple tasks from video chat to 4K livestreams, and it can even bring new capabilities to the table from Edge-powered augmented reality to smart infrastructure. But, the move from 4G to 5G isn’t as simple as flipping a switch. 5G networks and Edge computing require high-performance, open-architecture systems. To accommodate this transition, Supermicro is designing the backbone for 5G’s most advanced features. 

5G has a simple promise: deliver consumers high-speed data connections with massive bandwidth and low latency. 5G networks transmit vast amounts of information rapidly between devices and servers. Communications can go from one device to another in as little as a millisecond, and data transfers can hit speeds of up to 10 Gbps.

The improvements to speed and latency enhance the experiences already available on 4G LTE, but they also enable entirely new capabilities. Once 5G is commonplace, consumers will be able to stream video games to smartphones with high-quality graphics and negligible latency. Lightweight augmented reality devices will be able to offer a heavyweight experience. 5G networks can also have an impact on infrastructure, with more processing power to handle complex computing like traffic coordination.

Edge and cloud computing hardware is at the heart of all of those applications. Supermicro has various servers designed to fit the many needs along every point of 5G networks. The company has designed servers that are not just incredibly powerful but also highly efficient thanks to their inclusion of Intel® Select Solutions from 2nd Gen Intel® Xeon® Scalable processors to Intel® FPGA cards. Thanks to their efficiency, the servers have a low environmental impact and reduced energy consumption. Supermicro’s servers can also include leading hardware accelerator cards with GPUs for AI inferencing and FPGAs for advanced networking functions.

Supermicro has designed its servers to adapt quickly to the new needs of 5G, with smaller servers that can collectively form massive data centers. Supermicro’s Edge servers can even be built into rugged enclosures, increasing their lifetime in volatile environments. Supermicro has done just this with its customizable Outdoor Edge Systems.

These servers can withstand temperatures from -40℃ to 50℃, making them ideal for harsh environments. When the weather goes to extremes, a built-in, 300W heater and high-efficiency heat exchanger can keep the system up and running. The Outdoor Edge Systems are also protected against water and particles with an IP65 rating. Supermicro has further ensured they are GR-487-CORE and GR-3108-CORE compliant.

This all means that Supermicro’s servers can be deployed directly on 5G cell towers to keep data and processing power as close as possible to end users.

Supermicro’s Outdoor Edge Systems and SuperServers are powerful options for telecom and data center operators. Beyond their rugged design, they offer multiple PCI-E slots to enable GPU and FPGA acceleration or expanded storage capabilities. The promise of 5G is driven forward by Supermicro’s ability to put the hardware necessary for real-time AI inferencing, 5G RAN software, and much more right on the Edge.

As complex as these systems may all sound, Supermicro has refined the process of creating its 5G systems through its Building Block Solutions®. This enables simpler customization of each server to fit customers’ precise needs. So, where one system might be fully loaded with storage to deliver data as quickly as possible to end users, another might fill up the expansion slots with FPGA cards focused on a very specific workload. The 2nd Gen Intel® Xeon® Scalable processors can also flex to meet customers specific performance requirements.

Supermicro’s experience with open hardware and virtualization is uniquely suited to the needs of 5G. This ensures that every customer can find a solution that will flex to fit their needs. Supermicro already has a complete suite of server hardware available, but it’s continuing to work on even more 5G server infrastructure building blocks and solutions.

Its servers can bring storage closer than ever to users, delivering the fastest loading speeds. And, its hardware is ready to push AI forward through Deep Learning in data centers, increasing the capability of AI on its Edge servers. That AI could be tasked with everything from guiding self-driving cars to powering AR glasses, or even enabling advanced sensing devices for first responders.

If you want to prepare for 5G, you only have to look at what Supermicro is building. You can learn more about Supermicro’s 5G solutions here.

Supermicro might not be as known to gamers and enthusiasts as ASRock, Asus, or Gigabyte. But in the professional computing industry,the company provides solutions for many IT and server deployments. Reliability has propelled the company into a lucrative position in the data center.

Specifications

Features

On the flip side, Supermicro’s Professional Gaming brand attempts to deliver server-infused products into the enthusiast community. Though the previously reviewed X11SRi-IF had an unusual form factor and niche, Supermicro aims the C9X299-PG300F at the higher-tiered gaming PC or workstation near you. Sporting Intel’s X299 chipset, it supports Intel's i7 and i9 Core X-series processors with support for up to 165W TDP SKUs. Though it does support our Core i9-10980XE, it does not enable the additional four lanes equipped through the refreshed Intel processors.

Sporting a black and gun-metal finish, the PG300F in some ways looks a lot like some of the other X299 boards on the market. The 8-phase voltage regulator is cooled by a heatsink that looks similar to some older ASRock models, and the M.2 shield vaguely resembles some of MSI's earlier implementations. The I/O panel cover looks and feels like the plastic shrouds of old (MSI’s X370 Krait comes to mind), but beyond that, we start to see some of the server DNA beginning to show through.

Let's talk about PCIe. This board supports both 28-lane and 44-lane configurations, where the top two PCIe x16 slots are wired out for full-lane allotments (when available), and the bottom two can provide a variety of bifurcation (see table below). The PG300F also offers full bandwidth for both 80mm M.2 NVMe drives through the lanes provided by the PCH. The PCH further routes out 6 SATA ports, and depending on the processor deployed, one or both of the U.2 ports. However, when using the U.2_1 slot, the bottom slot gets reduced to an x4 connection.

Now let's dig into the server and workstation features. Intelligent Platform Management Interface (IPMI) is available for users that require remote access to the system, and Aquantia supplies its AQC107 hardware for 10Gb ethernet. You might also be curious about the VGA connector on the back, which is for viewing the UEFI and console output through the AST2500  baseboard management controller (BMC). COM, TPM, and chassis intrusion headers round out the other header interfaces on this board.

The back panel of the PG300F is straightforward, with two ports of USB2.0, USB3.0 Gen1, and USB 3.1 Gen2. The PG300F equips the Realtek ALC1220 chipset, and the standard five analog and one digital port are available for audio devices. The Intel I210-AT controller drives an additional 1Gb network connection.

Moving back to the board, six 4-pin fan headers are scattered evenly across the PCB, along with additional USB2.0 and USB3.0 Gen1 headers. Since this is technically a "gaming" motherboard, Supermicro provides two 12V RGB headers, which are programmable through the company’s SuperO software.

Overclockers will appreciate the Power, Reset, and Clear CMOS buttons located on the bottom edge of the board, along with the numeric debug display. One difference we see between the stock photos and our motherboard sample is the removal of the EQCD Samtec header below the 24-pin, which would be very helpful for low-level debug scenarios. Also, the stock photo shows a removable BIOS chip, whereas production units are not removable.

One deficiency for overclockers is the underwhelming voltage regulator heatsink. It’s sufficient, but our overclocking performance was a little spicy due to the mass of the heatsink.

Physically, this board is an ideal mix of convenience and features that leaves us with very few complaints. However, it could use a little bit more polish in terms of aesthetics and branding to make it stand-out from this firm’s server products, as well as competing offerings in the market. In short, Supermicro manages a decent job of blending the features of a workstation into a "gaming" motherboard with the C9X299-PG300F.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

The Supermicro PG300F motherboard's UEFI needs a bit of work to compete with the likes of MSI or Asus' consumer products. Don't get us wrong; the UEFI is simple to navigate and is functional for base-level operation, but after digging into the options for a week, we've identified a few sore spots.

EZ Mode is the default window upon hitting the DEL key at bootup and shows us boot order, DRAM status, high-level overclocking profiles, and general processor health. After activating the Advanced mode, CPU, memory, and other advanced menus are available on the left. We briefly perused the IPMI and Advanced menus. There you’ll find IPMI configuration, TPM, PCIe slot, boot, and other options for configuring the platform to fit your specific use case.

Also, the Advanced menu is where you will find most of the lower-level Intel CPU and chipset options such as CPU C States, Hyper-Threading and virtualization. We would have preferred more helpful information in the UEFI, and some of the formattings can be a bit weird.

For overclocking, our first stop is the CPU overclocking segment, with options for All Core OC presets, BCLK presets, and by-core max ratio limits. For the most part, tweakers can select either adaptive or override controls over various settings. Still, it isn't exactly clear which settings override others -- for example, BCLK Frequency and the BCLK Scale setting. Also, numeric options are highly configurable, but users will find that setting a 0 max ratio for core #1 will likely cause the system not to boot. In other words, make sure your settings make sense before saving and rebooting.

We had mixed success when tinkering in the Memory Overclocking section. XMP profiles work flawlessly, and our Corsair Dominator Platinum kit boots up with no problems. However, when tweaking timings, we noticed that subsequent reboots and entry into the UEFI wiped our previous settings. We have addressed this with Supermicro and hope to see the issue fixed in future UEFI versions.

Voltage controls are our last stop in UEFI checkout, and Supermicro doesn't do much hand-holding for users in this regard. We'll cover this in more depth in the overclocking section, but voltage levels can be as low as 0V to an uncomfortable 2.5V on some rails. With this in mind, we strongly suggest reading the manual to understand each setting and its limits. The documentation is an excellent resource and well worth saving to your hard drive.

This reviewer doesn't regularly install manufacturer's software, but it is necessary to install the SuperO Booster utility to program the RGB components of the system. The patterns and color selection are straightforward and required minimal tweaking in our setup.

On the CPU menu, we found that the frequency graph doesn't necessarily reflect the individual operating frequencies of our chip while running workloads bound to specific cores. Also, the overclocking functionality was less than accurate at times. When engaging single-threaded Cinebench runs, we often did not hit our desired overclocks. Again, read the manual if you intend to use this feature.

The Thermal menu is a nice feature to see while running the operating system, so that we can dial in our fan speeds while running an active workload. Beyond that, though, the voltage and memory menus aren't as attractive to us as we favor the UEFI, but they are there nonetheless.

Test Setup

Overclocking

Overclocking testing is where the Supermicro C9X299-PG300F begins to lose some ground. With mild success on other motherboards, setting a straight 4.8GHz all-core to overclock is not unreasonable and is our first step in testing. Immediately upon running AIDA64 at that setting with the Supermicro board, temperatures were well above 95C, causing minor heart palpitations. Digging into the UEFI, it appears that Supermicro defaults the VCORE setting when using the frequency presets to a massive 1.4V! After consulting with Supermicro, this is indeed the default option, and we strongly advise builders to set their clock speeds AND voltages on each boot. Skipping this step might cause permanent damage to your components.

Dialing back the voltage to a reasonable 1.25V, AIDA runs comfortably at around 80C while using our Fractal Design S36 360mm AIO. With additional thermal headroom, we hit 4900MHz without extra voltage for our extended-duration verification run. Hoping to catch the elusive 5.0GHz on this cooler, we increased the voltage to 1.325V, and temperatures approached the 100C threshold that we do not recommend for extended workloads. So, officially we hit 4900MHz at 1.25V and 82C with AIDA64 (pushing 82C on the VRMs).

Moving to the AVX-based workloads, thermals again became a problem, and we started dialing back the multiplier and voltages to compensate. Also, to keep things consistent with previous boards, we enabled AVX offset values of 3. Prime95 runs well at 4300MHz at 1.05V, which is only 100MHz behind some competitors.

As we alluded to in the UEFI discussion, memory overclocking with this board is both annoying and straightforward. Enabling XMP was our first step and worked fine out of the box. We could increase the data rates up to DDR4-3800 without issue or tweaking, but above that, we had to adjust our timings. Again, each reboot caused the timings to reset, which made this unsustainable. Fortunately, the DDR4-3800 ran at very tight timings compared to the other overclocks, which helped keep bandwidth results in line.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

The Asus board we tested continues to be a chart-topper, with an impressive 11% lead in memory bandwidth over the PG300F, while only besting it by 4.7% better latency.

Now with two samples tested with the EVGA RTX 2080 Super Hybrid, 3DMark Time Spy narrowly favors the Asus board, while the PG300F does perform better than both the ASRock and Gigabyte samples when it comes to CPU score. We observe similar behavior with Firestrike Extreme, but the physics scores appear to favor the ASRock Taichi CLX.

Moving to PCMark 10, we started to see some noise in the data with the previously tested boards scoring better with Essentials, but both Productivity and Content Creation leaned slightly in favor of the server DNA of the PG300F. The automated Microsoft Office suite rounds out the PCMark data, with the CLX narrowly beating the Supermicro C9X299-PG300F.

Gaming results today will focus on the RTX 2080 Super data. With Ashes of the Singularity: Escalation, the Asus ROG Strix refresh provides between 2 and 10 more frames per second (fps) across the resolutions and detail levels. It is interesting to note that the RTX 2070-equipped testbeds (ASRock CLX and Gigabyte Designare) still managed to place well at 1080p (when the CPU is the bottleneck), though the RTX 2080 is the better performer at 1440p.

F1 2017 runs very well on this platform, with only a narrow spread at the 1080P Medium settings, still managing to cap out 144Hz monitors at Ultra. GPUs aside, the Asus again pulls the lead here but only by a few frames at each of the settings.

The 18-core 10980XE is no slouch in the encoding department, and these timed workloads show tight timing margins across the tested motherboard samples. On average, the Supermicro PG300F slips a bit in this regard. but manages to get 2nd place in at least two workloads.

Aside from its name being similar to the current health crisis, the Corona 1.3 RT results here were mostly uneventful.

Cinebench shows very close results among the Supermicro, Asus, and ASRock boards, but the Gigabyte Designare 10G pulls ahead by between 2 and 6% compared to the lowest-performing competitor.

A new workload for our X299 analysis is Davinci Resolve rendering times, driven by the Puget Systems automated framework. This benchmark tests various video codecs and formats with different applied video effects and measures the total rendering time. Here, the Supermicro PG300F races past the three other tested systems but hits a snag when encoding the 4k RED footage.

Power and Temperatures

Supermicro leans heavily on its server-grade designs and manages to shave three watts at idle from the leader. but tacks on a bit at full load. Regardless, the average power consumption reported by the Kill-A-Watt is right in line with the ASRock CLX.

On the flipside, servers are designed to run in warmer environments, and the voltage regulator temps reflect that in spades. Despite ample airflow in our Thermaltake Suppressor F51 case, temps are 10 degrees warmer than the nearest competitor.

Aggregating all of our performance metrics, the Supermicro C9X299-PG300F manages to place second in overall performance due to its strong showing in productivity and creativity workloads, while slipping in timed applications and application startup. Above-average system performance and power consumption, in turn, place the PG300F in second place in efficiency as well.

Value and Conclusion

As we did in the last X299 motherboard review, we'll take into account system costs to account for the different GPUs equipped in the systems. Supermicro aggressively prices the PG300F at $440, which puts it within striking distance of the ASRock CLX Taichi. while saving builders up to $140 compared to the Gigabyte model. But GPU cost eats into those margins by quite a lot when building a complete system, forcing the Supermicro board into third place. Regardless, we think Supermicro did a good job of balancing features, performance, and price, which make this board a decent option for those looking to build a workstation.

But that leads to the trouble we have here. Out of the box, the product works as designed and performs well compared to others on the market. However, the UEFI experience and the fact that it’s aimed at gamers just make it hard to compare against brands that have had many product cycles in this market segment. Just as Asus managed to polish its X299 ROG Strix, maybe future additions to the Professional Gaming Brand from Supermicro can refine the company’s formula while still delivering on value.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

It’s giving season here at Tom’s Hardware, so we have another exciting giveaway lined up for our readers. This time we’re giving away a tempered-glass PC case from Thermaltake and a mini-ITX motherboard from Supermicro!

Our grand prize package has everything you need to get started on your holiday PC build. Up for grabs is the Thermaltake Core P1 TG Mini ITX wall-mount, tempered-glass chassis and the Supermicro C9Z390-CG-IW mini-ITX motherboard motherboard.

For the full lowdown and instructions on how to enter, head to the giveaway thread in the Tom’s Hardware forums.

No purchase necessary. Void where prohibited. Giveaway is only open to legal residents of the fifty (50) United States, the District of Columbia, and the United Kingdom, 18 or older. For a complete list of rules please see the Terms and Conditions on the Gleam Giveaway Widget. This giveaway is open until January 1st, 2019. 

Update: As of Nov. 15, single units are still available only in UK.

The “more is better” philosophy of extreme computing has brought everything from CrossFire and SLI graphics arrays to workstation-class motherboards to the consumer PC market. So we were hardly surprised when companies started producing dual-motherboard cases (and even a desk) that lets users to install an ATX or EATX gaming platform on one side and a Mini-ITX number-cruncher or streaming PC on the other. But Supermicro has a thought for those of you who want real number crunching prowess on the Mini-ITX side: Why not go Xeon?

Born of a customer-exclusive design that Supermicro got permission to take public, the X11SRi-IF was originally intended to fit four complete configurations into a 2U chassis. For the consumer, that simply means that this board can support an LGA 2066 Xeon and four full-sized ECC registered DIMMs into the compact space meant to hold a Mini ITX consumer board.

Specifications

Features

 The first thing that makes this board frustrating from a consumer standpoint:  You’ll need a Xeon and ECC DRAM simply to use the X11SRi-IF, as the board won’t even start with an HEDT CPU such as the Core i9-7900X. And attempting to pair that Xeon with a non-ECC desktop UDIMM returns an error that only proves that these parts are electrically compatible, but not firmware supported.

Supermicro sent a CPU that would compete directly against our Core i9-7900X, the Xeon W-2155, along with four p/n MTA36ASF4G72PZ-2G6H1RQK sticks of Micron’s 32GB ECC registered DDR4-2666. While we can’t yet match the memory capacity with a consumer kit, the workstation processor runs the same 3.30 GHz base frequency and 4.50 GHz top boost frequency as the Core i7-7900X. Heck, the two CPUs even have the same 10-core, 20-thread configuration and 13.75 MB of L3 cache.

Another thing that isn’t consumer friendly (even for “prosumer” workstations) is an I/O panel that has only two USB3 Gen1 ports. This, on a board that doesn’t even have any internal USB headers. On the other hand, prosumers who are good at networking can do nearly everything that normal people do over USB via one of the two Gigabit Ethernet ports. We said two ethernet ports because the third RJ-45 connector is for remote system management via IPMI (Intel’s “Intelligent Platform Management Interface”).

The lower corner does support an old-fashioned serial port but, apart from the Power/Reset/HDD LED/Power LED header, there are no front panel connections. Not even audio, because the board doesn’t have a codec. And if you’re using this with a graphics card, the paucity of USB ports and lack of additional PCIe slots means it’s not getting audio, unless you use the graphics card’s HDMI audio pass-through. At least there’s that odd option.

What the Supermicro board does do better than most consumer boards of this size is connect three fans, rather than two. Three should be considered the minimum since most compact performance cases have a front, rear, and CPU fan, but we’ll understand if you’re willing to use splitter cables.

Lo and behold, this board also has a single 512MB DDR4-2400 integrated circuit (IC) serving its AST2500 Server Management Processor, an ARM11-based part that includes 2d integrated graphics. You’ll find a jumper on the bottom edge of the board to disable that feature.

While there’s no visible SATA, the metal-shielded header seen in the image above supports both PCIe x4 and 4-port SATA cables. Compatible data cables include Supermicro’s CBL-SAST-0933 four-drive SATA and its CBL-SAST-0956 NVMe (using SFF-8639 for U.2). Supermicro also takes that unusual step of providing SATA power via the two white four-pin connectors seen on the X11SRi-IF’s forward edge, each of which supports its CBL-SAST-0886 dual-drive breakout cable.

Expanding outward, we find an identical NVMe cable header between the CPU socket and I/O panel VGA port. This header is NVMe only, allowing the board to support two of the firm’s CBL-SAST-0956 data cables.

A third hit to the X11SRi-IF’s suitability for desktop users comes from its misplaced upper rear standoff, which was mandated by the board’s DIMM placement. We had to remove the standoff from our test system to prevent shorts, and our case didn’t have an alternative hole to match the X11SRi-IF’s unusual configuration.

You may have noticed that the X11SRi-IF has no 24-pin connector. But it does have a power button header: For solo use, an ATX-style power supply with EPS12V cable must be used with its power-on pin shorted to a ground pin, and power savings with the system off but the power supply on relies upon the power supply’s idle features. However, what seems off-putting to builders of compact PCs might be a blessing to those who build two systems within a single case, as some of those cases have only one power supply mount.

The EPS12V header feeds a 6+1 phase voltage regulator rated at 165W continuously, in addition to all onboard components including the SATA power cable headers. Once again, what sounds like a great configuration for a specific type of server can also be a perfect match to dual-motherboard systems.

The X11SRi-IF’s back side supports a single 2280 M.2 SSD upon a low-profile (4mm-tall) connector, similar to the back-side slots of other Mini-ITX boards.

The X11SRi-IF doesn’t support any overclocking features, so we tossed in a few dozen screenshots of the things its firmware does support, many of which are remote management features.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Benchmarks

Supermicro wants you to believe that its X11SRi-IF is unique in its support for a 140W LGA 2066 Xeon, but ASRock did the same thing for the Core i9 version of this CPU two years ago in its X299E-ITX/ac. Since the Xeon W-2155 and Core i9-7900X are supposed to be identical from the perspective of our benchmarks, this leaves the two boards to compete based on other parameters, which primarily relate to the type of memory used.

The X11SRi-IF requires ECC-registered DIMMs, and the ones we’re using have twice as many ranks and twice as many banks per rank as a standard 8GB UDIMM. While both features can dramatically improve memory performance, the same ECC technology that improves data accuracy can also put a damper on performance. Though we didn’t have a set of four non-ECC modules at the same capacity, we used a set of SODIMMs that defaulted to the same DDR4-2666 rating on the X299E-ITX/ac to keep things as fair as possible.

Sandra, 3DMark and PC Mark

We lead with memory performance just to see how badly ECC would impact its performance, or how much the extra ranks and banks would help. It’s obvious here that the performance advantage of doubling the ranks per channel and doubling the banks per rank far outweighs the performance deficit of verifying parity bits via ECC.

Looking at a few scores from 3DMark and PCMark, the Supermicro X11SRi-IF also appears to hold up its processor’s higher Turbo Boost ratios a little longer than the X299E-ITX/ac. Remember, both the Xeon and Core i9 have the same Turbo Boost rating.

3D Games

You probably won’t be using a Xeon for gaming, but if you do, you’ll be happy to see that the X11SRi-IF outperforms the X299E-ITX/ac in both Ashes and F1 2017. The later benchmark is known to be somewhat dependent on memory performance, particularly at its lower quality settings.

Mixed Workloads

The X11SRi-IF squeaks out a win in 7-Zip, but the X299E-ITX/ac solidly thumps it in Cinebench. The two appear evenly matched in our classic timed tests.

Power, Heat and Efficiency

The X22SRi-IF pulls much less power at idle than the X299E-ITX/ac, but the difference mostly goes away at full load. We had to test both with Prime95’s less-stressful “Large FFTs” because neither could hold both processors up to their rated Turbo Boost ratios when running the stressful Small FFTs test.

We should also note that our particular power supply can idle down to around 1W with the X299E-ITX/ac turned off, but the power supply can’t be turned off completely when connected to the X11SRi-IF and instead pulls up to 15W in standby mode. With different power supplies treating such a low-load condition differently, your results will vary.

The X11SRi-IF’s CPU ran a little cooler, but its voltage regulator a little warmer, using our Fractal Design Celsius S24 closed-loop cooler with fans atfull speed to provide the voltage regulator’s required airflow.

The X11SRi-IF performed a little more than 2% better on average than the X299E-ITX/ac, and based on our previous memory test we’re betting that most of that gain should be credited to the manufacturer’s choice of memory. We simply didn’t have a set of 8GB single-rank ECC DIMMs from which to produce a more closely-matched configuration.

Final Thoughts

Our performance-per-dollar chart is complicated not just by the fact that the X299E-ITX/ac has far more features or that the X11SRi-IF supports server processors and DRAM, but that Supermicro doesn’t yet have a US retailer for the board. Priced around £320 in UK tax included (and $330 US before tax), Supermicro is currently offering any distributor that wants these a low price of $256 in quantities of 2000. Thus, we used $300 as a best-estimate.

Pricing doesn’t get any easier for the X299E-ITX/ac, because this current model was always intended to be a limited release, and the sellers that still have these have tacked on a 50% surcharge for the privilege. We had to turn back to its original $400 MSRP simply to get back to basics.

The X299E-ITX/ac probably has $100 worth of extra features, and it supports a far cheaper processor model to provide the same number of cores and frequency. Yet for those who want ECC and advanced remote management, the X11SRi-IF’s feature set will appear superior.

In the end, it’s the lack of consumer-friendly I/O that has us considering the X11SRi-IF only as a prosumer number crunching platform, and not a primary PC. Server guys will be happy to hear that, but most end users at this point will probably just go with something AMD for a plethora of cores at a reasonable price.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

A famed hardware leaker known as @momomo_us on Twitter tweeted a diagram of what appears to be Supermicro's upcoming X12DPi-N motherboard as well as some interesting benchmark results of an unidentified 14-core, 28-thread Intel processor.

Intel's upcoming Whitley server platform for Cooper Lake-SP and, subsequently, Ice Lake-SP processors has been rumored to support the PCIe 4.0 interface for a while now.  

AMD has already adopted the PCIe 4.0 on its consumer Ryzen processors, so Intel is already late to the game. At this rate, it's hard to tell if PCIe 4.0 will arrive on an Intel consumer or server platform first. 

The motherboard diagram also shows the LGA4189 CPU socket, which TE Connectivity designs and produces for Intel. There are two variants to the LGA4189 socket, but we can't tell which version the X12DPi-N motherboard is using.

With two sockets, the Supermicro X12DPi-N was shown with a pair of mysterious Intel processors with 14 cores, 28 threads, 17.5MB of L2 cache and 21MB of L3 cache. The chip seemingly has a clock speed of 3 GHz and was working in tandem with 256GB of DDR4-3200 Micron-branded RAM.

It's uncertain if the 14-core chip belongs to the Cooper Lake-SP or Ice Lake-SP family (both support the LGA4189 socket). 

However, the L2 cache could be the key to solving the mystery. The processor seemingly sports 1.25MB per core, which doesn't match any configuration that we know (Skylake-SP and Cascade-SP Lake feature 1MB per core). Since Cooper Lake-SP is practically placeholder until Ice Lake-SP arrives, the chips will likely maintain the same L2 cache design. 

With the shift from Skylake to Ice Lake (14nm to 10nm), Intel increased the L2 cache from 256KB per core to 512KB per core. Now that the enterprise side is making this same 14nm to 10nm transition, there's a chance that Intel could upgrade the L2 again. Assuming the Cooper Lake-SP sticks to 1MB per core, then Ice Lake-SP arriving with 1.25MB per core seems possible. So if we had to guess, we'd say the 14-core, 28-thread chip is probably from the Ice Lake-SP family.

AMD held its European launch in Rome, Italy of its new EPYC processors. Along with announcing that the company's EPYC Rome data center processors have now attained 100 world records (and counting), the company also announced a slew of new OEM servers.

AMD also unveiled its EPYC 7H12, a 64-core 128-thread that boasts a beastly 280W thermal design power (TDP) envelope, which allows the processor to reach a 2.6 GHz base and 3.3 GHz max boost frequency, marking the highest performance of its Rome product stack. 

AMD designed the new chip, which requires watercooling to extract the ultimate performance, for high performance computing (HPC) workloads. ATOS unveiled its new Bullsequana XH2000, a hybrid supercomputer designed for exascale-class supercomputers that supports the new 7H12 chips.

The system supports the EPYC 7H12, cooling it's 32 1U blades (per rack) with an advanced water cooling solution, and said that it provides up to 4.2 TFLOPS of performance per chip, making the 7H12 up to 11% faster than the current top-of-the-stack EPYC 7742 processor.

As a quick refresher, EPYC's 7nm process brings density advantages over Intel's 14nm, which equates to higher core counts. It also brings power reductions (albeit with a 12nm I/O die caveat) that lead to more work done per watt (a critical consideration in the data center), higher clock speeds, more cache, and ultra-competitive pricing. Pair that with inherent cost and yield advantages of a chiplet-based design, a revamped Zen 2 architecture that brings an ~15% uplift to instructions per cycle (IPC) throughput, a fast move to PCIe 4.0, and an industry-leading serving of memory channels and throughput for x86 processors, and EPYC is no longer seen as the Intel "alternative." Now it's perceived as the leader in terms of leading-edge features that attract the heavyweights of the industry, as evidenced by the explosive uptake of Rome in the HPC and supercomputing space.

But general purpose servers see the lion's share of volume shipments, so having a full stack of enterprise-class systems, with support for the latest software and backed by OEM service contracts, is important to assure broad uptake. AMD has been on a full court press working with the leaders of the server industry to assure that it has a broad and capable stack.

Dell EMC recently announced its full lineup of enterprise-grade servers in both ultra-dense single-socket servers, which is a considerable focal point for AMD because its servers offer far more cores than Intel's Xeons in this form factor, and dual-socket models. Dell EMC also used its PowerEdge R7515 to set a new TPCX-V world record, notching a 280% performance improvement over its previous-generation EPYC server.

That record joins others posted by AMD's partners (not AMD itself), with the tally now reaching up to 100 world records for AMD's EPYC Rome processors. AMD also touted other solutions, such as systems from Supermicro that leverage EPYC's leading PCIe 4.0 connectivity by supporting a healthy ration of GPUs, and announced initiatives with IBM Cloud, Nokia, and OVHcloud, all of which join an ever-expanding ecosystem of Rome systems. AMD's foundry partner TSMC also made an appearance to discuss the company's close partnership with AMD and its joint R&D initiatives on future products. 

AMD also provided some comparative figures that provide a broad outline of EPYC Rome's value proposition relative to Intel's processors. As with all vendor-provided benchmarks, we should take these figures with a grain of salt, but there is no doubt that the EPYC Rome processors pressure Intel on multiple axes. Whip in broad vendor support with systems that feature enterprise-class support for both the hardware and software, and it's clear that EPYC Rome is emerging as a potent foe to Intel's Xeon lineup.

Researchers from security company Eclypsium have uncovered a set of USB vulnerabilities in in the baseboard management controller (BMC) on Supermicro’s server boards (models X9, X10 and X11) that could allow attackers to hijack thousands of servers. The researchers, the same ones who warned earlier that Supermicro's servers can be easily backdoored, named the vulnerabilities USBAnywhere.

Taking Over Supermicro’s Servers Remotely

BMC is a 'computer within a computer,' much like Intel’s often-criticized Management Engine (ME), that allows IT administrators to remotely control and update computers on a network. Normally, the BMC is locked within a network, so that it can’t be remotely accessed from outside of said network. However, not everyone does that, sometimes on purpose, if they would like to access some servers over a web interface. 

These interfaces aren’t typically designed with security in mind, either, which makes it that much easier for attackers to find existing vulnerabilities and exploit them. In this particular instance, the Eclypsium researchers found a vulnerability in the BMC feature that allowed IT admins to remotely mount images as USB devices. 

This could be exploited because it essentially gives hackers the ability to take over a device remotely as if they had local USB access to the machine. For instance, the attackers could install a new operating system or implant malware via the remote USB access.

BMC Vulnerabilities

The Eclypsium team found four vulnerabilities. These flaws included included using plaintext authentication and unauthenticated network traffic, weak encryption for the remote connection and an authentication bypass flaw in the Supermicro X10 and X11 platforms that would allow new clients on the virtual media service to use an old client's permissions. 

The Eclypsium researchers noted:

"If a valid administrator had used virtual media since the BMC was last powered off, the authentication bypass vulnerability would allow an attacker to connect even without the proper username and password.”

The security team that uncovered the bugs believes that tens of thousands of Supermicro servers could be open to this attack. The Eclypsium researchers said they contacted Supermicro, which has issued a fix for its server motherboards, but Supermicro’s customers will still need to install the update for protection against USBAnywhere attacks.

Enter for your chance to win the Supermicro Supero C9Z390-CGW Motherboard​.

Geared for the performance users who need the basics and the flexibility of all the latest technologies within a single motherboard, the SUPERO C9Z390-CGW appeals to the hardcore gamer. With gigabit wired and wireless networking, multi-channel audio, and RAID ready storage options, you will be ready to build your next gaming PC.

To enter head to the giveaway page and follow the instructions.

No purchase necessary. Void where prohibited. Giveaway is only open to legal residents of the fifty (50) United States, and the District of Columbia, 18 or older. For a complete list of rules please see the Terms and Conditions on the Gleam Giveaway Widget. This giveaway is open until September 6, 2019.

Hey there Tom's Hardware readers! From time to time we try to give back to the community with a giveaway. It's a win-win: You get free stuff, in this case the Supermicro C9Z3890 motherboard, and we get warm fuzzy feels. This mini-ITX motherboard has all the bells and whistles. It features the Intel Z390 chipset, supports up to 32GB of DDR4-3866 RAM, and has an M.2 Slot for all that SSD goodness. 

For your chance to win, follow the instructions in our giveaway thread on the Tom's Hardware forums. Good luck!

Last year, Bloomberg ran a report, saying Supermicro-supplied servers come with Chinese backdoors and that this may have been a reason for Apple to dropped them in 2016; although Apple denied espionage concerns at the time. Although new research publsihed today doesn’t exactly confirm Bloomberg’s report that Supermicro servers ship with pre-installed backdoors, it does point to the microcontrollers used by Supermicro and the firmware that comes with them being easily backdoored without detection. 

Researchers from Eclypsium, a firm specializing in firmware security, were able to commission a bare-metal server from IBM, install a backdoor in one of its microcontrollers, and then allowed IBM to re-use the server for other customers. The researchers were later able to reclaim that same server and noted that the backdoor was still active on the server, which means IBM lacks proper reclamation process that can clean previously used bare-metal servers of accidental or intentional backdoors. Attackers could use the same process that the researchers used to brick or steal data from other IBM customers.

Supermicro's “Parasitic Servers” Are Easily Exploitable

Previous research had shown that baseboard management controllers (BMCs), which are motherboard-attached microcontrollers, can give extraordinary remote access to servers inside data centers. The management capability on these BMCs is provided via the Intelligent Platform Management Interface (IPMI), which in many ways is similar to Intel’s Management Engine and its Active Management Technology and poses the same large risks of allowing attackers to take over servers remotely.

The IPMI is supposed to allow organizations to make configurations to a large number of servers remotely, even if the servers are turned off. In 2013, researchers pointed out that these BMCs can create “parasitic servers” that could allow attackers to take over entire server fleets in data centers.

Dan Farmer, a white hat security researcher, said at the time about BMCs and IPMI:

“Imagine trying to secure a computer with a small but powerful parasitic server on its motherboard; a bloodsucking leech that can't be turned off and has no documentation; you can't login, patch, or fix problems on it; server-based defensive, audit, or anti-malware software can't be used for protection; its design is secret, implementation old, and it can fully control the computer's hardware and software; and it shares passwords with a bunch of other important servers, stores them in clear text for attackers to access.”

Back then, the vulnerable firmware for these BMCs was developed by ATEN Technology, a Taiwanese company. These BMCs came pre-installed on servers from Dell, HP and other providers. A new paper is now showing that IBM’s servers are also highly vulnerable to undetectable backdoors installed in SuperMicro’s BMCs.

What Compromise of a Server's BMC Means

When an attacker compromises a BMC in a server, this opens up a variety of attack scenarios. For one, servers could be bricked, which means attackers could blackmail cloud service providers into paying them money “or else,” just as DDoS attackers have done in the past.

Having such low-level access to all the servers that are connected and controlled via a BMC means that the attackers can also extract all sorts of data from those servers from many cloud service customers. Ransomware and potentially crypto-mining malware could also be installed on many of the compromised servers.

The Eclypsium researchers recommended cloud service providers extend the reclamation process to the firmware level to ensure that any bare-metal reclaimed servers from a previous customer weren’t intentionally compromised.

The reclamation process should also include updating the BMC firmware manually, as well as the UEFI firmware via the BMC. The service providers should also continuously monitor changes to their servers’ firmware to spot attacks as they happen.

Over the past few years, more companies have come to realize that supply-chain security is just as important if not more important than applying software patches. Verifying that purchased hardware hasn’t been tampered with either from factory or somewhere in the supply chain should be an even bigger priority for cloud service providers who are responsible for the data protection of millions of customers.

Supermicro has reassured shareholders that a third-party investigation refuted a report from Bloomberg saying its motherboards had been compromised by Chinese intelligence agencies. The company said "the investigations firm found absolutely no evidence of malicious hardware on our motherboards," and with that being the case, it believes "today's announcement should lay to rest the unwarranted accusations" made about its products.

The claim that Supermicro's motherboards were compromised was published shortly after a Bloomberg Businessweek feature story entitled "The Big Hack: How China Used a Tiny Chip to Infiltrate U.S. Companies." Bloomberg said in its followup report that a telecommunications company security expert named Yossi Appleboum "provided documents, analysis and other evidence of the discovery" after "The Big Hack" was published.

Those reports were published on October 4 and 9. Supermicro CEO Charles Liang released a statement refuting Bloomberg's reporting on October 18, saying "no one has shown us a motherboard containing any unauthorized hardware chip" and explained that the company was "undertaking a complicated and time-consuming review to further address the article." The letter published today likely stems from that review.

Supermicro has not provided a lot of information about the review process, but that isn't uncommon among security companies. Most of the firms that investigate data breaches, hacks and other security problems specifically tell clients not to disclose their relationship to the public (which is why so many cite an "independent review," "third-party audit" or the like).

Bloomberg's "The Big Hack" and subsequent reporting immediately raised concerns throughout the U.S. after publication. But in the time since, many of the companies named in the report have denied its veracity. Apple CEO Tim Cook said his company wanted the story retracted, and Amazon publicly decried the report.

Bloomberg's news story about Supermicro's letter notes that "Bloomberg Businessweek has previously said that it stands by its story." At the time of writing, the publication has not made an any other statement about the situation.  

Features and Layout

It’s amazing to consider that a form factor as small as Mini ITX has found its way into the high-end enthusiast space. These small motherboards were originally built for simple tasks like running Internet kiosks or digital signs. However, today we have much smaller computers for those tasks and enthusiasts can competently cram a high-end consumer SSD onto a Mini ITX board with no problem. 

ASRock is the first company to properly power and cool Intel's Core i9-9900K in this compact factor with its Z390 Phantom Gaming-ITX/ac. Packed with features for such a small board, it's also surprisingly affordable. When we wrote this it was selling for about $190 (£145).

Specifications

Enthusiast motherboards these days are often overloaded with RGB. So the presence here of a mere three ground-effects-style RGB LEDs on the Phantom Gaming-ITX/ac’s underside is modest by those standards--but we're not complaining. Features include an oversized voltage regulator heat sink that’s designed to support real overclocking, dual M.2 slots and even a Thunderbolt 3 controller. All that said, keep in mind that this is still a somewhat value-priced product, so it only has a single Gigabit Ethernet port and a half-speed (20Gbps) Thunderbolt 3 controller. But that’s still more features than the limited on-board lighting and sub-$200 would suggest.

The Z390 Phantom Gaming-ITX/ac even kicks its Wi-Fi up to 1.73Gb/s, and does so inexpensively by using Intel’s new CNVi PHY to interface with the integrated controller of the Z390 chipset.

Aside from connections for the included Wi_Fi antennas, the back is loaded up with a CLR_CMOS button to the left, a PS/2 port for legacy peripherals, two USB 3.1 Gen1 ports, plus DisplayPort 1.2 and HDMI 2.0 for integrated graphics. The fun doesn’t stop there, as the Type-C connector located between four USB 3.1 Gen2 ports supports graphics output in addition to data (at 20 Gbps combined). Oh, and buyers still get Intel Gigabit Ethernet (I219V, to be precise), along with five analog audio jacks via Realtek’s high-end ALC1220 codec, and optical digital audio output.

ASRock has finally realized that even in the gaming PC market, Mini ITX buyers need (or at least want) two M.2 slots. The first M.2 slot is located inside the PCH heat sink, its heat spreader being semi-integrated into that sink. A heat pipe connects the chipset and voltage regulator sinks.

Builders can still use the M.2 slot located, by ASRock Mini ITX tradition, underneath the board. Both M.2 storage slots support PCIe 3.0 x4 and SATA interface drives, but while the top slot supports 60mm and 80mm drives, the bottom slot fits 80mm drives exclusively.

The Phantom Gaming-ITX/ac supports a slightly wider range of connections than the Z370 version it replaces, but it still has only 30 HSIO lanes and is still connected to the CPU by a four-lane DMI. That means, among other things, that leaning heavy on both M.2 drives at the same time will force them to compete for bandwidth. It also means that ASRock had to remove something to make way for the integrated USB 3.1 connections. The Phantom Gaming-ITX/ac has only four SATA ports (compared to the Z370 version’s six), and one of those is shared with an M.2 slot’s SATA interface. But the lack of a USB 3.1 Gen2 front-panel header will likely be a larger issue for builders who’ve picked a case that incorporates that feature.

Other documented headers include three 4-pin fan headers and one Addressable LED header on the top edge of the board. Also there live RGB LED, beep-code speaker, USB 3.0, an Intel-style front-panel button/LED headers, as well as a chassis intrusion header, along with front-panel audio and USB 2.0 headers at the back. Interestingly, there's an undocumented 8-pin header (in white) to the right of the board's 8-pin CPU power header, and another undocumented 9-pin header (in black) just behind the front-panel button/LED group.

The Z390 Phantom Gaming-ITX/ac accessory bundle is fairly light. In the box you'll find two SATA cables, an I/O shield, a Wi-Fi antenna, a driver disc, and documentation.

While the chipset’s limitations may make the use of a PCIe x2-based Thunderbolt 3 controller understandable, the seven-phase voltage regulator appears to be the Phantom Gaming-ITX/ac’s weakest specification. We’ll put that theory to the test later on in our overclocking evaluation.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Software and Firmware

Following the tradition it set with Fatal1ty F-Stream, ASRock’s A-Tuning gets yet a third name and color scheme with the Phantom Gaming Tuning app that's included with this board. It still bases its auto-tuning algorithm on existing overclocks, and the board lacks an EZ OC profile for our CPU. Therefore the app's Start button is inactive by default. Manual tuning from the board's OC Tweaker menu still works, however.

The Z390 Phantom Gaming-ITX/ac uses a 1.35V DRAM setting as default, even though our memory specified 1.20V for its DDR4-2400 default configuration. Phantom Gaming Tuning’s System Info page read a voltage of 1.376 to 1.384V, so we checked and found that 1.384V was closer to reality. In fact, we had to drop the firmware setting to 1.310V just to get a range of 1.348V to 1.352V, according to our voltmeter.

ASRock Polychrome RGB isn’t exactly configured to show the available lighting of this motherboard model, and instead wrongly labels the three LEDs on the underside of its bottom edge as being on the “PCH Heatsink.” The lighting controls work, but all three LEDs are on a single color channel, so rainbow effects aren’t available. The software is also compatible with our RGB memory, but clicking the “Sync” button drops color pattern options to “Breath”, “Blink”, and “Static” single-color modes.

Firmware

Like other ASRock boards, the Z390 Phantom Gaming-ITX/ac's UEFI defaults to its EZ Mode interface, uses the keyboard’s F7 function to switch into Advanced mode, and includes a setting under the Advanced mode Advanced menu to change the default GUI. While EZ Mode allows changing from default to XMP memory settings, boot order, fan profile, and even has a “Tools” menu for firmware update modes, you’ll need to switch to Advanced Mode to do anything as complex as disabling the onboard RGB lighting. Moreover, Advanced Mode includes ASRock’s OC Tweaker menu, with CPU, DRAM, and Voltage settings, in addition to five user profiles for which a set of custom CPU, DRAM, and Voltage settings can be stored. Oh, and if onboard storage somehow isn’t enough, users can transfer these profiles to a USB flash drive and share them with their friends.

Our overclocking begins with a CPU ratio of 49X at a BCLK of 101 MHz. That’s the same 4,949 MHz fixed clock as we’ve reached on other boards, and a 5% overclock from the Core i9-9900K CPU’s reference 4.70 GHz all-cores-loaded Turbo Boost ratio. Of course it’s still a bit less than the 5GHz available on up to two cores under light loads by default.

The Z390 Phantom Gaming-ITX/ac was the first board to give us hope for a 5.0 GHz full-load overclock using a mere 1.30V. It did manage to hold up for about 40 minutes before we saw a blue-screen error. Dropping to 4,998MHz by using 49 x 102 MHz just wasn’t enough of a decrease to get past the same (roughly 40 minute) crash time, despite several attempts.

“More voltage” is of course the trick to achieving overclocking stability, which is why Intel allows this CPU to reach up to 1.36V when using its two-core 5.0 GHz “Turbo Boost” setting. But keeping all eight cores cool at such high voltage levels is extremely difficult. We chose 1.30V to keep temperatures in check, and while our 240mm closed-loop cooler has a little extra capacity for higher voltage, we’re not changing our standards to accommodate one board.  Maintaining that voltage under changing loads was as easy as keeping the “Level 1” CPU Load-Line Calibration setting that the board automatically applied after we set the multiplier and core voltage.

Since there is no such thing as a perfect 1.350V, we’ve chosen a DRAM voltage limit of 1.355V for our memory overclocking comparison and prevent cheating via higher voltages by measuring the slots with a voltmeter. The Z390 Phantom Gaming-ITX/ac pushed an incredible 40mV over the set voltage. Worse still was that the extra voltage was obscured by a fictitious reading within the board's H/W Monitor report. The “1.308V” report shown above corresponds to a measured 1.348 to 1.352 volts at the slot while using the board’s 1.310V setting.

All that said, the Z390 Phantom Gaming-ITX/ac didn’t need the extra voltage to get a good data rate: Even after correcting its over-voltage, the board pushed our DDR4-2933 to DDR4-3818. But we’re actively trying to discourage a never-ending cycle of motherboard manufacturers sneaking increasingly greater voltage levels into their DIMM slots, only to have memory manufacturers respond with modules that need the extra voltage, prompting motherboard manufacturers to use even more voltage until the CPU’s integrated memory controllers eventually gives way at so-called “stock” settings.

ASRock’s Tool menu includes an outgoing email client for tech support messages, a tool for copying RAID drivers to a thumb drive, an SSD Secure erase tool, an Instant Flash firmware update mode, Internet Flash firmware download mode, and Network Configuration to help users reach the Internet Flash server prior to installing an OS. Internet Flash did reach the server this time and return a report of “no update available,” but a check of the motherboard’s webpage showed that an update was indeed available. After downloading the update manually on another PC, we used Instant Flash to install it, despite the Internet Flash feature insisting it didn't exist.

Only one of the three Z390 Phantom Gaming-ITX/ac fan headers provides voltage-based RPM control. Profiles can be manually configured by percentage or adjusted graphically under the FAN-Tastic Tuning sub-menu, and factory-programmed settings can be modified to match your specific fan’s profile using the Fan Tuning algorithm.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

How We Test

The second Mini ITX board we’ve tested using the Core i9-9900K, ASRock’s Z390 Phantom Gaming-ITX/ac goes up directly against Supermicro’s C9Z390-CG-IW. Full ATX models we included in our comparison give the tiny boards some big competition while helping to fill out the charts.

The same platform that cooled the 10 cores of our Core i9-7900X worked equally well with the eight cores of the Core i9-9900K, as we’ll show in the overclocking evaluation on the next page.

Comparison Products

Benchmark Settings

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Benchmark Results and Final Analysis

Most enthusiast-class motherboards employ a type of overclocking that ignores the CPU’s lower all-core Turbo Boost mode and instead deploys its highest single (or two) core Turbo Boost ratio regardless of the load. While most boards currently don’t do that with the Core i9-9900K, we still make sure the setting is disabled prior to our basic performance and power tests. We also enable all of the CPU’s power-saving features in firmware and leave the memory in its default (non-XMP) mode.

Synthetic Benchmarks

The two Mini ITX board here support only half of our four-DIMM RAM kit, and each module has one rank of 8GB. The Intel memory controller’s preference for at least four ranks of memory had us guessing about the performance impact of using only two ranks on the Z390-CG-IW. But things still look good in 3DMark for the Z390 Phantom Gaming-ITX/ac. It’s not that there’s no performance impact, but the impact is small.

Sandra Cryptography is bandwidth-restricted and impacted by the reduced ranks of the two-DIMM boards, as further illustrated in its Bandwidth benchmark. On the other hand, the performance gap we’d noticed on the competing Supermicro board went away for the Z390 Phatom Gaming-ITX/ac in Compubench.

3D Games

Now that we’ve seen it on a second two-DIMM board, we understand that giving up half of our memory ranks has some impact on all of our games. Though it affects a wider variety of games on the Core i9-9900K than we’d noticed in earlier 8700K tests, the difference appears smaller in the notoriously affected F1 2015.

Timed Applications

Handbrake completion times continue to rise even after we restored our benchmark image. We can only speculate that the SSD may be slowing after repeated reloads. Other than that triviality, the Z390 Phantom Gaming-ITX/ac performs nicely.

Power, Heat, & Efficiency

The Z390 Phantom Gaming-ITX/ac makes enormous strides in energy use, holding the 4.70 GHz eight-core Turbo Boost setting at a mere 1.16V and returning a 60W decrease in power consumption compared to the full-ATX Z390 ASRock Taichi. Conversely, the C9Z390-CG-IW appeared to use Intel’s 5.0 GHz two-core-boost voltage even when running an 8-core load at 4.70 GHz.

Lower voltage levels usually mean cooler temperatures, but we were still a little concerned about the Z390 Phantom Gaming-ITX/ac’s smaller voltage regulator and heat sink here. But it turns out there was no need for our concern.

Not only was the Z390 Phantom Gaming-ITX/ac the lowest-energy board in the mix, there was only a 0.8% performance deficit attributable to its use of only two modules from our four-DIMM kit. The result is an amazing efficiency score that's 18.7 percent above the average. And while buyers who are only willing to purchase two single-rank DIMMs could save a few watts on one of the larger boards, doing so would cancel out that 0.8-percent performance difference.

Overclocking

Everything we said at the top of the page about enabling power saving features goes out the window in overclocking. We’re seeking the highest 100-percent stable frequency while using Prime95 small-FFT’s to apply a 100% load across all eight cores and sixteen threads.

We mentioned in the firmware section that the Z390 Phantom Gaming-ITX/ac was the first board to run our CPU at 5.0 GHz for a full 40 minutes with all eight cores and 16 threads fully loaded, but that we had to give up on that setting after being unable to get it stable for much more than 40 minutes. We should also mention that the voltage regulator forced thermal throttling after around 20 minutes, and that we had to switch from a  roughly 60 CFM to a roughly 160 CFM fan to prevent that from happening. Dropping it back to 4.95 GHz was also necessary to get the voltage regulator temperatures under control. so this really is close to the limit of the stock voltage regulator cooler.

Memory overclocking was the only test we did on all of the big boards in both two and four DIMM configurations, and getting bandwidth numbers from Sandra is part of that process. This version of the bandwidth chart shows how each of these boards looks with two DIMMs at max O/C, and the Z390 Phantom Gaming-ITX/ac appears very competitive. Previous charts showed the 4-DIMM boards getting nearly 40GB/s when all the slots were filled.

Final Analysis

The Z390 Phantom Gaming-ITX/ac only needed to beat the other Mini ITX board Z390 board we've tested so far to declare a victory. It did so in performance, power consumption, heat, efficiency, and overclocking. Though most of the features between these boards are evenly matched, the ASRock board leads by having a Thunderbolt 3 controller--even if the controller only runs at half-bandwidth (20Gb/s). Of course, chipset pathway limitations meant something had to give, and the Z390 Phantom Gaming-ITX/ac lacks the front-panel USB 3.1 Gen2 header of the Supermicro C9Z390-CG-IW. On the other hand, if you have an older case, the ASRock board has the USB 2.0 header that Supermicro left out. Both boards have front-panel USB 3.0 headers.

Even if you never use Thunderbolt 3, it’s hard to deny the supremacy of a board that does everything else better than the competition while costing less. The Z390 Phantom Gaming-ITX/ac is even cheaper than the non-Wi-Fi version of MSI’s MGP Z390 Gaming Pro Carbon, though that won’t matter to anyone who needs the extra DIMM and PCIe slots afforded the ATX form factor. But for those in the market for a Mini ITX board to support Intel’s Core i9-9900K, the Z390 Phantom Gaming-ITX/ac is easily our first choice. 

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Features and Specifications

How do you solve a problem like that of the Intel Core i9-9900K's demands? Adding two extra cores to the Coffee Lake architecture meant it should automatically convert an extra 33 percent of power into heat. But Intel didn’t stop there. It also reverted to old-fashioned solder to join the CPU's core to its heat spreader, helping to eliminate the heat but doing nothing about the power requirement. And that worked so well that Intel decided to allow overclocking. With a core voltage limit that exceeds what we use to overclock, it’s hardly a shock that some boards come up a little short here. But before you write off the C9Z390-CG-IW, you’ll want to read the full story.

Supermicro C9Z390-CG-IW Specifications

Cramming a bunch of cores into a small space is nothing new, as we saw in our X99 and X299 Mini-ITX reviews, but the new Core i9-9900K is a different beast. First of all, hardly any enthusiast motherboards are power throttling it to its rated 3.6GHz to get it under its rated 130W thermal envelope under heavy loads. Second, most of the motherboards with an LGA 1151 socket that support it are based on previous designs meant only to handle the prior top-mainstream model, the Core i7-8700K.

We never got the opportunity to test the predecessor to Supermicro's new C9Z390-CG-IW, but our review sample's 7-phase voltage regulator certainly looks the part of a previous-generation mainstream board. Tiny sinks on those transistors (MOSFETs) do little to convince us of its Core i9-pushing power, but we’re hoping for a surprise larger than that of the 12-LED background RGB that spills from under the board’s leading edge.

Come to think of it, we already spotted one surprise in the board's M.2 storage slot, which fits neatly over the cutout section of its PCH sink. Most Z-series Mini-ITX boards have just one M.2 slot on the back, but the C9Z390-CG-IW has slots on both its back and front. And while both PCIe 3.0 x4 slots are designed to hold 80mm M.2 drives only, the rear one adds an SATA interface to support legacy drives.

The I/O panel includes a PS/2 port for legacy peripherals, two USB 3.1 Gen1 ports (aka USB 3.0), DisplayPort 1.2 and HDMI 2.0a, four USB 3.1 Gen 2 ports (including one Type-C), Gigabit Ethernet via Intel’s i219V PHY, 1.73Gb/s Wi-Fi via Intel’s 9560 CNVi PHY, six analog audio jacks and a digital optical audio output. A couple more USB ports would have been handy for power users, but we instead find a jumper in the panel’s only vacant space.

The jumper allows the board to force the PCIe x16 slot to function in x8/x8 mode for use with double-slot riser cards, which sounds incredibly practical coming from a company that cut its teeth on server parts. Making the switch still requires that the CPU support PCIe bifurcation, but that feature is common to all Intel's Core i9, i7 and i5 CPUs that use the LGA 1151 socket.

A few of the C9Z390-CG-IW’s headers are designated for factory use or undocumented, but there are USB 3.1 and 3.0 front headers located above and before the top-side M.2 storage interface, along with front-panel audio in front of the I/O audio jacks, a TPM header in front of the PCIe x16 slot latch, as well as an RGB strip and two fan headers on the top edge. Oddities include an OC mode jumper under the PCIe x16 slot’s lower edge, an SATA Disk-On-Module power header in front of the TPM header and a Supermicro-specific front-panel LED/button section between the two fan connectors that includes an output for an overheat/fan failure LED.

The C9Z390-CG-IW includes a printed manual, driver disc, I/O shield, cable label stickers, two Wi-Fi antennae and two SATA cables.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Firmware

Recent years of developing a consumer-friendly UEFI has resulted in an EZ Mode GUI from Supermicro, along with advanced settings available via the keyboard’s F7 function. Users unfamiliar with overclocking can still set boot order and enable XMP memory profiles from EZ Mode. Charging straight into Advanced mode, we found an Overclocking menu that presented CPU, Memory, Graphics and Voltage submenus.

The CPU Overclocking submenu includes basic overclocking configurations for:

• 4.3 through 4.6GHz using a 1.3V CPU core, 1.3V System Agent and 0.95V CPU I/O
• 4.7 and 4.8GHz using 1.35V CPU core, 1.30V System Agent and 1.2V CPU I/O
• 4.9GHz at 1.4V CPU core, 1.35V System Agent, 1.2V CPU I/O and Level 4 Load Line Calibration
• 5.0 through 5.5GHz using 1.45V CPU core, 1.4V System Agent, 1.2V CPU I/O and Level 4 Load Line Calibration

Since our CPU overheats at 1.35V, we started with the 4.6GHz setting and worked our way up to 4.94GHz at 49 x 101 MHz.

Default power limits were insanely low, and even overclocking modes don’t appear to be designed with an 8-core, 16-thread CPU in mind. We increased both CPU power limits (second image above) to the motherboard’s limit to prevent overcurrent protection from resetting the board.

The DRAM conundrum should feel familiar to old-time overclockers: Setting XMP profile blocks out individual timings and data rate. Switching from Default to Custom memory profile allows users to tweak everything from Default baselines, and switching from XMP to Custom memory profile allows users to tweak everything from XMP baselines. Unfortunately, users who don’t know what to set for all 66 adjustments aren’t likely to get very far, as only seven of those can be configured to automatically adjust based on a manually selected data rate.

The Load Line Calibration feature adjusts voltage upward in response to it dropping under heavy software loads. We found that the easiest way to get 1.3V CPU core when using 16 threads of Prime95 (small FFTs) was to choose a 1.285V core setting and Level 2 Load Line Calibration. Concerning Load Line Calibration, Level 1 put our core voltage through the roof, while Level 3 allowed it to drop well below our desired operating level.

The non-overclocking menus for CPU and Memory settings allow users to make non-overclocked adjustments. We enabled all of the CPU’s power-saving features for comparative benchmarks and power measurements but not for overclocking evaluation.

RGB control is found within the Boot Feature submenu of the Advanced menu. However, "on” and “off” are its only modes, and the only “on” profile is a fast-flashing rainbow mode.

The C9Z390-CG-IW has only two fan headers, and both of those support PWM-based settings only. Automatic modes appear to control a three-pin RDM1225S, despite the PWM-based settings of manual profiles.

The Save & Exit menu includes everything from boot settings to two registers where users can save their firmware configuration as profiles. The board defaults to AHCI storage mode and must be manually configured to access UEFI-enhanced OS installations.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

How We Test

The C9Z390-CG-IW is our first Mini-ITX motherboard designed to support Intel’s new 8-core, LGA 1151 processors, so we have only ATX boards to use for comparison. The ASRock Z390 Taichi, MSI MPG Z390 Gaming Pro Carbon and Gigabyte Z390 Aorus Master fill those roles.

The same platform that cooled the 10 cores of our Core i9-7900X worked equally well with the eight cores of the Core i9-9900K, as we’ll show in the overclocking evaluation on the next page.

Comparison Products

Benchmark Settings

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

Benchmark Results and Final Analysis

We enabled all CPU-based power saving features in firmware and disabled fixed-mode Intel Turbo Boost manipulations prior to our basic performance and power tests. The maximum Turbo Boost ratio for the Core i9-9900K is 5GHz with up to two cores loaded, and that drops to 4.7GHz when more than four cores are loaded.

Synthetic Benchmarks

While some boards enhance advanced memory timings to get a small advantage in a few benchmarks, the C9Z390-CG-IW suffers from the fact that the boards its being compared to have four DIMM slots. Given that Intel processors consistently show a performance preference for a minimum of four memory ranks and that each module of our four-DIMM kit is single-rank, we’ll cut it a small break.

The C9Z390-CG-IW started out behind the four-DIMM boards at 3DMark’s lowest effort test but gradually caught up as the workload increased. Sandra Memory Bandwidth shows the deficit was due not to a problem with the board, but the CPU’s preference for more memory ranks. The mix put it on par with the others in Cinebench but a stroke or two behind the pack in Compubench.

3D Games

Gaming benchmark history would have us believe that the two-DIMM C9Z390-CG-IW should fall farthest behind at medium settings in F1 2015 and take a minor hit in Ashes of the Singularity, but that’s not exactly what happened. The “good enough” score in Ashes was a little farther off than expected, and the big hit in F1 2015 came at its ultra preset. Perhaps the newer CPU is pushing us to the limits of the graphics card and DRAM?

The oddest result was in Metro Last Light Redux, which our previous configuration showed only took a performance hit when saddled with memory slower than the baseline of these modules. Our expectations are slowly evolving to accommodate the newer CPU.

Timed Applications

Where less time means more performance, the C9Z390-CG-IW keeps us very close to schedule. However, we did see a 30-millisecond loss in PowerPoint.

Power & Heat

This is where things get weird. Sometimes, the C9Z390-CG-IW would shut down when we started Prime95 small-FFTs. Other times, it would shoot up to around 242W before permanently throttling to 3.4GHz, well below the CPU’s rated frequency. Setting a higher power limit in firmware occasionally allowed us to reach 4.7GHz, where we eventually found a 344W full-load power reading. Most of the time, the higher power limit would push the voltage regulator to its critical limit, causing it to power off. It wasn’t until we turned Intel Hyper-threading off that the voltage regulator stayed at full load long enough to get a proper VR thermal reading. Even then the CPU was only pulling 297W.

A default voltage range of 1.32 to 1.36V for the CPU core was the only data we could find to explain the power throttling issue and high wattage and CPU temperature. In fact, the CPU temperature crossed the board’s default threshold of 100 degrees Celsius (212 degrees Fahrenheit), which resulted from the motherboard’s default setting of 125 degrees Celsius (257 degrees Fahrenheit) being out of range for the CPU. Since the CPU’s thermal limit is 115 degrees Celsius (239 degrees Fahrenheit), restricting the motherboard via a 10-degree thermal offset actually enabled the 100 to 115-degree range.

The great news is that performance was down by only 1.9 percent, despite the performance deficit our CPU encounters when switching from four to two ranks of memory. The C9Z390-CG-IW performed more competitively than we would have expected from a two-DIMM board.

Overclocking

By now you’re probably wondering “If this board was tripping its own circuit breaker at stock CPU settings with power-throttling disabled, how can it be overclocked?” The board does have an OC-mode jumper, but enabling that boosts CPU core voltage to 1.48V, tripping the board's circuit breaker all the faster. But you may also remember that we said it was running up to 1.36V CPU core at stock settings. We actually use less voltage to achieve our overclock!

At 1.3V CPU core, the C9Z390-CG-IW will run all day at 49 x 101 MHz under Prime95 small-FFTs. That’s 16 super-AVX-loaded threads! Could this be the first board we recommend exclusively for overclocking and never for use at the board's default i9-9900K settings?

Even though the C9Z390-CG-IW didn’t make the necessary adjustments to secondary and tertiary settings to properly overclock our DDR4-2933, it did let us run those settings at the same 101MHz base clock. (If you've noticed that the charted bandwidth advantage of the four-DIMM boards is reduced from previous reviews, that's because we’re showing their two-DIMM results.)

Final Thoughts

Our C9Z390-CG-IW review was a wild ride, primarily because the board's default settings couldn’t run our CPU at its full 4.7GHz 8C/16T rating under Prime95. The board throttled our CPU below its non-turbo-boosted 3.6GHz at stock settings and often tripped its thermal protection mechanism with power throttling disabled.

The C9Z390-CG-IW still looks good in a performance-per-dollar chart, and it’s the first Z-series Mini-ITX board we’ve seen in a while with two M.2 storage slots. The biggest problem is that we really can’t recommend it to anyone who wants to run Prime95 small-FFTs on the Core i9-9900K. That may not be a big deal to you, but our reviews require consistency.

Our findings around disabling Hyper-threading (see Power & Heat, above) is especially relevant for those with a Core i7-9700K, since that CPU doesn’t have Hyper-threading. The C9Z390-CG-IW had no trouble running our 9900K with Hyper-threading disabled, and the 9700K presents an even lighter load by being at least 100MHz slower.

For i9-9900K ovecrlocking, you could just overclock at a reasonable voltage, such as the 1.3V we chose. The 4.94GHz we reached under an 8C/16T load still beats the stock 4.7GHz by 5 percent. And if you care enough, we did try setting a -50mV offset at stock frequency settings and came up with a 1.22V CPU core drawing 256W at 4.7GHz in Prime95 small-FFTs. The fix is there, but applying it is up to the buyer.

MORE: Best Motherboards

MORE: How To Choose A Motherboard

MORE: All Motherboard Content

From Duplo Dragons to Giant Chassis

We saw plenty of award-worthy products here in Taipei and filed dozens of news posts from the show floor. But just like at any big trade show, there were several eye-catching, sometimes quirky things that fall outside the bounds of our usual coverage, because maybe they aren’t quite brand-new, they aren’t strictly products you will be able to buy, or they fall a bit outside of our usual coverage areas.

They're still well worth highlighting, so we’re recognizing them here where we’ve picked the 21 coolest things we saw (and didn't give awards to) at Computex 2018.

Tesoro Gram XS

If Dave Bowman from 2001 was shopping for a mechanical keyboard, he'd get Tesoro's futuristic Gram XS. The keyboard has a great stark-white aesthetic with flat keys and a super-slim profile that make it look like it belongs on Discovery One. Vibrant, customizable RGB lights shine through the flat, chiclet-style keys.

Despite its slim dimensions, the keyboard felt pretty good when I ran briefly tried typing on a model with the company's own clicky blue switches. If you like a quieter keyboard, you can get it with red switches. You can even buy it in black -- the same boring color as most keyboards -- but I'm sorry Dave, I'm afraid I can't do that. The Gram XS already on the market and available for $119. -- Avram Piltch

InWin Z-Tower

Case maker InWin is known for showing off crazy concept chassis at trade shows—and often bringing them to market. Recently, they’ve been high-tech and motorized, looking like something from a Sci-Fi effects shop. But the company’s latest, the Z-Tower, is more akin to a work of modern art.

Made of nearly 90 pounds of cast aluminum in a sweeping, almost organic-looking spiral and standing several feet tall, it was the one case at Computex 2018 that was impossible to ignore. If you think you’d like to build your next PC inside the Z-Tower, be prepared to make lots of room—in your home and your bank account. The company says the Z-Tower will likely sell for several thousand dollars. -- Matt Safford

Lian Li Desk PC (with Dual Computers)

Meet the ultimate piece of geek furniture. Lian Li has been making desks that double as PC cases for a few years, but this year's show was the first time I got to see them up close. The company showed off two upcoming models, one that holds a single computer and another with dual chambers so you can have one PC that plays games and another that streams (or a server) in the same desk. Both models have hydraulic lifts that you can adjust to turn this into a standing (or kneeling on the floor) desk.

There are USB ports and a power button on front lip. A clear glass surface lets you see all the glory of your components at all times. Most importantly, the entire top is free for your monitors, keyboard, mouse and other stuff, with no space devoted to a tower case. -- Avram Piltch

Cougar Minos One

Any time you add a screen to a device that doesn't normally have one, that's a win. Cougar's Minos One is one of two mice with color displays that I saw at Computex, with the other being Coolermaster's MM830.While the Coolermaster's pointing device has a great feel, the Minos One has a much larger panel on board.

When I saw the Minos One on the show floor, its screen was showing an animated Cougar logo, but a Cougar rep told us that, when this product launches later this year, it will be able to show real-time information like the DPI setting or in-game statistics. This wired mouse also has two custom RGB lighting zones and the ability to run at up to 4,000 dpi. -- Avram Piltch

Corsair Obsidian 1000D Case

In 2018, the hot question is no longer "how many Cores does your computer have" but "how many computers does your computer have?" There are other chassis on the market that hold two discrete systems inside, but Corsair's Obsidian 1000D is one of the biggest.

At 27 inches tall, 27 inches deep and a full 12 inches thick, the Obsidian 1000D is large enough to make your enemies quake with your fear and your frenemies light their RGBs green with envy. Ideal for gaming on one motherboard while streaming on the other, the case has room for 18 different fans and four 480mm radiators. It also provides space for up to five 3.5-inch hard drives and up to six 2.5-inch SSDs.

You can buy the Obsidian 1000D now for $499. -- Avram Piltch

Lian Li Strim RGB Cable

If your fans have RGB lights, your power supply has RGB lights, your RAM has RGB lights and even your power supply has RGB lights, why not your cables? Lian Li's upcoming Strim is a customizable RGB cable that runs from your power supply to your motherboard.

There's also a version that goes from your PSU to your graphics card. If you're going to bling out your system, you've gotta go all the way. -- Avram Piltch

Gamdias Hades P1 RGB and Nyx P2 RGB

Despite the inconvenience, most hardcore gamers prefer the accuracy and low-latency of a wired mouse to a wireless one. However, with Gamdias's Hades P1 RGB, you get the best of both worlds; a 12,000 dpi wireless mouse when you want it and, if you plug in its USB cable, a direct wired connection to your PC. More importantly, this attractive pointing device has customizable RGB lighting and a choice of side panels you can pop on or off to change your grip.

The Hades charges wirelessly using Qi technology and that's where Gamdias's other really-cool upcoming product fits in. The Nyx P2 RGB mouse pad not only has a Qi charging pad built into its upper left corner but also features an attractive ring of customizable RGB lights. Both products are due out later this year. -- Avram Piltch

Thermaltake H200 RGB Headset

We didn't have a headset award category on Tom's Hardware this year, but based on my brief experience with Thermaltake's H200, I'd award the company $40, the MSRP, of my own money to buy one. On the outside of the over-the-ear pieces, you'll find customizable 3D RGB lights that really pop.

Meanwhile, the padded ear cans felt extremely comfortable when pressing against my ears. They seemed to be just the right size: not so small they have to go in your ear and not so large that they smush your lobes. I didn't get to listen to the output, but my colleague, Sherri Smith heard some rich sound coming from the device. For me, the comfort, looks and reasonable price puts this device on my personal wish list -- Avram Piltch.

Patriot Viper V765 Keyboard

Gamers want switches that go up and down quickly so they can shoot or move faster, but many typists, including yours truly, can't live without clicky keys. Patriot's upcoming Viper V765 is one of the first keyboards to use Kaihl Box White switches, which have a lower actuation point than blue switches, but still have that snappy feel and pleasant click sound.

When I tapped the keys on the Viper V765, they felt like they could offer the perfect balance between speed and tactile feel. Typing comfort matters most, but this keyboard also has great looks, thanks to its aluminum chassis and fully-customizable RGB backlighting. It even comes with a comfy wrist rest that attaches or detaches magnetically. -- Avram Piltch

Corsair iCue Software

Every company that makes RGB peripherals or components has its own control software. However, Corsair's new iCue application takes lighting customization to the next level. iCue gives you incredibly fine grain control over each LED in your system;it lets you create different effects for every key on your keyboard or even every single light inside every single one of your case fans.

Better still, iCue can change the light show based on what happens in the game you're playing. It was really cool to see the demo system's case, keyboard and mouse all flash yellow when the main character caught fire in Far Cry 5. So far, Far Cry 5 is the only game that works with iCue, but hopefully more are coming. You can also set your lights to react to your system temperature, so they show that your CPU is "red hot." -- Avram Piltch

ASRock Optane 905P M.2 RAID Array

RAID 0. When you absolutely, positively have to get ridiculous transfers speeds, accept no substitutes. Now, what happens when you take a bunch of Intel's blindingly-fast 905P M.2 SSDs and put them into a RAID array? ASRock decided to find out by connecting four of the drives and connecting them together using a VROC card.

The results are truly impressive, with a sequential read rate of 10.7GB per second and  9.7GB per second of write speed. True, the 4K random reads were worse for the RAID array than for a configuration that has just one disk. Just don't put this array anywhere near your Thunderbolt port, because it's so fast that it may take you on a journey back to 1955. -- Avram Piltch

The Inception Project (RGB PC Wall)

You can think of it as everything that’s wrong with modern PC trends, or PC art on a massive scale. But it was impossible not to stop and stare at this mesmerizing wall of liquid-cooled, RGB-lit PCs on display at the Nangang Convention Center. I can say this with certainly because nearly every stick of RAM, AIO cooler, and chassis I saw at Computex sported its own similar rainbow light show.

Yet, at least twice during the long week of endless meetings and writing, I found myself standing mesmerized in front of Inception Project. Maybe it was the scale of the whole thing, or maybe my brain just needed a break from the madness of the show. -- Matt Safford

Ducky Year of the Dog Keyboard

There were plenty of interesting new keyboards at Computex. But Ducky’s limited-edition Year of the Dog mechanical clacker managed to catch my eye from a couple booths away without the use of bright RGB LED lights. (They’re there embedded in the Cherry MX keys, but they weren’t lit when I was ogling the keyboard).

Designed with the help of Hong Kong tattoo artist Michael Chan, and covered with art based on Chinese mythology, this may be the first keyboard that deserves to live in an art gallery. That said, if you’d like to take one home you’d better be ready to order come August or September. Ducky says that’s when the Year of the Dog Keyboard will go on sale, but only 2,018 will make their way off the assembly line. I’m sure fans of Asian gangster cinema--or just fans of tattoo art--will gobble them up quick.  -- Matt Safford

Well Buying Industrial’s Briefcases Full of LED Switches

The corners, crevices, and back walls of trade shows are often the best places to find niche products that are intriguing--or just plain odd. Well Buying Industrial (Co., Ltd) had a small booth full of all kinds of LED switches. Now, there’s nothing all that strange about switches. But the presentation, with dozens of models all lit up inside a collection of briefcases, made it seem as if they were going to be sold door-to-door. -- Matt Safford

MSI’s Massive Duplo Lucky Dragon

No trade show is complete without spotting Lucky, MSI’s dragon mascot. The company has taken to bringing a huge stuffed Lucky to CES.

But this massive model at MSI’s headquarters, made from Duplo bricks and standing at least 10-feet tall, was a truly pleasant surprise. It’s tough not to smile when you bump into a room-sized plastic cartoon dragon. I’m just glad I didn’t have to help put Lucky together. -- Matt Safford

Noctua Desk Fan Prototype

Noctua makes some of the best fans in the PC business, thanks in large part to serious engineering and research into the finer points of airflow. With this prototype, the company is taking its expertise outside of the PC realm and onto the desk, in a serious endeavor to keep you cool during those hot summer days.

Using the company’s newly redesigned NF-A12x25 fan, plus a cone designed to maximize and adjust the airflow passing through the fan, the device delivered a surprisingly strong breeze from such a small device. It was just what I needed after stepping out of Taipei’s sticky humidity and onto the show floor. A Noctua rep told me this desk fan will very likely make it to market, though there will be some aesthetic and design changes, along with a USB power cable so you can run the fan from your PC or a power bank. -- Matt Safford

Noblechairs Hero

This year, Computex looked like it was half tech expo, half furniture store, because so many different vendors showed off their new gaming chairs. Noblechairs upcoming Hero model stood out from the crowd by focusing on comfort, ergonomics and a refined aesthetic.

Unlike most companies, which use cheaper components like wood or bonded leather in their models, all of Noble's chairs have a robot-generated metal frame and your choice of durable materials such as polyurethane or real leather.

Due out within the next few months for an estimated starting price around $450, the Hero is the executive chair I want for my office. Forget about play time, because this chair is just as good for working long hours editing a report as it is for all-night gaming sessions. It features built-in, adjustable lumbar support rather than a pillow and everything about the chair feels premium, from the tautness of the material to the tight hinges and dials you use to adjust it.

I had a chance to go butts-on with the Hero at Computex and, rather than leaning back, it made me want to roll up to a desk and start typing. This chair could actually make you more productive. -- Avram Piltch

MSI Gaming Premium Stack (GPS)

Headset stands are the next frontier for innovation. At Computex, I saw several different stands with USB hubs in them, RGB lights and even surround-sound speakers. However, MSI's Gaming Premium Stack, aka GPS, stands out from the crowd, because it has a holographic screen on it.

When it comes out later this year for an estimated starting price of $399, the Stack will feature a stand, an optional wireless charger and the Lucky Box, which is a holographic screen that shows animations or system information floating in a 3D glass tube. MSI showed us the control software which lets you choose between a couple of default animations of company mascot Lucky the Dragon or show stats such as your computer's system temperature. You can also upload your own animated GIFs. -- Avram Piltch

ASRock Superb Mining Rig with Seven Phantom RX 580 Cards

ASRock’s recently launched Polaris graphics cards are so new, I’d never actually seen one in person before the show. And yet, here sit seven Phantom RX 580s tightly packed into an “ASRock Superb Mining” rig. Perhaps the only thing more superb than the hash rate of a rig like this: the money it could have made mining, had ASRock found a way to produce these cards in 2017, before the current coin mining slump. -- Paul Alcorn

Noctua Chromax Heatsink Cover Mod

I’ve seen all sorts of case and component mods over the last couple of decades, first as an enthusiast, then as a journalist. But these painted heatsink covers for the Noctua Chromax NH-D15 cooler line really stood out on a show floor dominated by glass and RGB. Created by Portuguese graffiti artist Le Funky, these covers show off the possibilities of creating a truly unique heatsink setup if you decide to trick out the heatsink covers the company sells for a few of its coolers.

Now, I’m not saying a custom-painted cooling cover would be beneficial for your CPU temps. But if you’re after some standout aesthetics for your next build and you have an artistic streak, it’s probably worth trading a few extra degrees of load temperature for a cooling tower that’s truly unique. -- Matt Safford

Supermicro Dual-Socket Gaming System

This week, we heard a lot about an exotic server processor and platform repurposed for the desktop PC, but at least Supermicro is straightforward about it. Supermicro's new dual-socket gaming system is designed for the 0.05% that want the absolute most power they can get for a gaming system.

This dual-socket system supports two Xeon Scalable processors of your choice, but you'll need one with high clock speeds to push this gaming system along. For the most well-healed, you could slap in two Xeon Platinum 8180's, which weigh in at $10,000 apiece, and create a  56-core 112-thread monster. Supermicro also touts ECC support as a feature, but I'm not sure how that's relevant to gaming, as it certainly won't give you a boost. In either case, I'd sure like to take this rig home, but I'm coming up about $20,000 short. -- Paul Alcorn

LAS VEGAS, NV -- Supermicro had its new flagship C9X299-PG300 motherboard on display in its suite at CES 2018. Notably, the company specifically listed support for Intel Core X-Series CPUs with up to a 300W TDP. Currently, the 165W TDP Core i9-7980XE has the highest TDP rating of the X-Series, so the 300W rating is an interesting wrinkle. We asked Supermicro for more details, but the company said it couldn't share any more information. 

Supermicro may not be the first name you think of when it comes to gaming motherboards, but the company is looking to change that. Supermicro knows, much like the other vendors, that capturing the enthusiast's eye, and their build, requires a strong focus on aesthetics in addition to the normal overclocking bells and whistles. Striking that balance isn't easy, and the company previously tried a less-conventional route with a bright green and black color scheme. It has since discarded that scheme in favor of the black and grey motif seen on the C9X299-PG300.

The X299 motherboard comes in the standard ATX form factor and features eight DDR4 slots that Supermicro says supports 4,000MHz+ after tuning. You also get six SATA 3.0 ports and two U.2 ports, along with two M.2 slots hidden under the long black extensions between the PCIe slots. These beefy covers have thermal pads underneath to help cool the SSDs.

The motherboard features dual LAN with 1 and 10 GbE connections, with the faster connection coming courtesy of an Aquantia AQC107. The -PG300 sports a debug LED and two 8-pin ATX power connectors that feed the 8+2 power phase underneath the beefy heatsinks. The board also features four PCIe 3.0 x16 slots and a single x1 slot, but you won't find any LED lighting on the motherboard. Instead, Supermicro provides two customizable RGB LED headers so you can roll your own solution. The company says that it is working on more elaborate lighting for future product generations. We also see the USB ports on the rear, including a USB Type-C port.

Supermicro also had the enthusiast-oriented C9X299-PGF on display, which features somewhat pared down specifications compared to the -PG300 but includes IPMI 2.0 support for out-of-band management. We don’t typically find that workstation/server-class feature on desktop PCs, so it makes for an interesting differentiator. The cooling solution for the VRMs isn't quite as robust with this model, but it only supports X-Series processors up to 165W TDP. You also get 5GbE connectivity with this model.

Supermicro's 300W rating for a Core X-Series CPU certainly raises many questions. We've heard of the 165W Core i9-7980XE pulling up to 550W with beefy watercooling and even closing in on 1000W with LN2, so it's no doubt the processors are power hungry. Intel's thermal paste chokes off most of the fun early, though. It's fun to imagine a 300W TDP processor coming our way, but it could be nothing more than marketing. Supermicro representatives wouldn't provide more information.

Supermicro's C9X299-PGF is already on the market. Its beefier C9X299-PG300 counterpart will join it later in Q1.

LAS VEGAS, NV -- The M.2 specifications have been great for consumer use, but lack some features large enterprise customers want in their pizza box server. The enterprise equivalent to M.2 is the U.2 specification, but that's built on old 2.5" disk drive form factor technology. Flash storage has moved beyond traditional form factors with success, but the consumer M.2 specification doesn't carry over well to the enterprise.

The two M.3 SSDs we've seen thus far from Samsung and now Adata feature a wider (30.5mm vs 22mm) width that gives manufacturers more surface area to mount components. The extra surface area gives the companies more options with NAND packages, host power failure capacitors, and larger controllers.

Adata's IM3P33E1 features a PCIe 3.0 x4 host connector that support hot plug. The drive will enter the market in sizes that range from 240GB to 1.92TB. This is lower than Samsung's, which reaches up to 16TB, but much more economical. Adata claims up to 3,200 MB/s sequential and 1,800 MB/s sequential write speeds over the NVMe 1.3 protocol.

The connector in the server doesn't use an angle when inserting the drive. This direct, straight through connector gives users hot plug capabilities without the use of a complicated mechanism. The only server we've seen with the M.3 form factor puts 34 drive sleds on the front of the 1U chassis. The server consolidates the drives vertically and fits more high-speed storage than possible with 2.5" U.2 ever could in 1.75" height server.

The pin out is the same between M.2 and M.3, but don't expect to use these in your desktop anytime soon. The two M.3 drives we've seen from Samsung and Adata use 12v instead of 5v. Even the name M.3 is not a technically correct term at this time—it's a specification that does not exist. Samsung first used the M.3 name at Flash Memory Summit 2016 but later shifted over to Next Generation Small Form Factor (NGSFF) after catching some heat from the committee that oversees the M.2 specification. With more than one company showing NGSFF drives, we should see some movement on the specification soon.

Introduction

Intel's Xeon E5-2600 v4 finds the company at the intersection of a waning desktop market and the exploding data center segment. Less momentum on the desktop side is naturally worrying; but being a leader in the high-growth enterprise space is a great place to be; Intel just happens to sell 99 percent of the world's processors destined for the data center. There is no single contender that poses an immediate threat to the company's dominance, though IBM's Power architecture and a budding Qualcomm initiative in China may gain ground over the long term.

The latest evolution in Intel's Xeon line-up is made possible by the Broadwell-EP architecture, manufactured on a 14nm process. That's a shrink of Haswell-EP's 22nm lithography. Many see Broadwell-EP as Intel's last attempt at satisfying the famed tick/tock cadence.

No doubt, Moore's Law is slowing to a crawl. The first precursors became evident in July of last year when the company indicated it'd push back its 10nm process to 2017. A recent 10-K filing merely formalized the obvious and inevitable.

Intel's filing indicates that it is migrating from the familiar tick/tock tempo to a slower process, architecture and optimization cadence. It appears that the challenges of shrinking transistors are becoming too difficult, and too expensive, to circumvent at such a rapid pace.

The new rhythm will lengthen the amount of time the company utilizes its 14nm and 10nm processes, which makes good business sense. Financially, it is better for Intel to recoup more of its investment into increasingly expensive tooling while extending its profitability window for each processor generation. There is no clear threat to the company's dominance, so its technological leadership position isn't at risk.

The Xeon E5-2600 v4 family wades into this new reality with a number of improvements that extend beyond more cores and cache (though it includes those as well). Perhaps that'll gives us some insight into Intel's plans moving forward.

Intel Xeon E5-2600 v4 Series

The Broadwell-EP microarchitecture increases the maximum number of cores/threads from 18/36 to 22/44, and also makes room for up to 55MB of shared L3 cache (up from 45MB). Intel still enables four channels of DDR4 memory, but increases the peak data rate to 2400 MT/s (a 15 percent improvement). Intel also added new memory features, such as support for 3DS LRDIMMs and DDR4 Write CRC (an enhanced form of error control). 

We're told the CPUs benefit from an IPC increase of 5.5 percent or so through a series of optimizations we'll cover on the following page, and are socket-compatible replacements for Xeon E5-2600 v3-based Grantley systems (LGA2011-3). Existing motherboards/servers will require a BIOS update, though. The existing C610 series chipset soldiers on, meaning that most platform features remain unchanged. You still get 40 lanes of PCIe 3.0 and two QPI 1.1 ports, for example.

The E5-2600 v4s offer a different base frequency and Turbo Boost setting for AVX and non-AVX functions, but Intel notably allows each core to operate in either mode without affecting the clock rate of other cores. In the past, every core ran at a lower base and peak Turbo Boost frequency if the cores were running a mix of AVX and non-AVX code. That restriction is no longer in place.

MORE: Best CPUs
MORE: Intel & AMD Processor Hierarchy
MORE: All CPU Content

Our Xeon E5-2697 v4 sample features 18 Hyper-Threaded cores and provides a non-AVX base frequency of 2.3GHz, peaking at 3.6GHz under Turbo  Boost. Running AVX code, the base drops to 2GHz, while Turbo can still take you to 3.6GHz.

The Product Stack

The fourth-generation E5-2600 series features 22 models designed for a wide range of workloads. Intel groups the processors into High Core Count (HCC), Medium Core Count (MCC) and Low Core Count (LCC) segments ranging from four cores and 10MB of LLC to 22 cores with 55MB of cache.

New Intel 3D NAND SSDs

All of the processing power in the world is worthless if your CPU is waiting on the storage subsystem. Intel helped foment the SSD revolution when it introduced its first datacenter-oriented drives. Of course, the move was a strategic one. Intel makes more money selling storage. In turn, SSDs help unlock the potential of multi-core processors, encouraging more Xeon business.

Intel and Micron produce NAND in the jointly operated venture called IMFT, which recently announced 3D NAND-based products. Intel chose to launch two drives based on the technology, its DC P3520 and DC P3320 SSDs, in tandem with Broadwell-EP.

The DC P3320 series is news to us, but the DC P3520 is not. We discovered a document back in August that foretold the release of Intel's DC P3608, and it also included pertinent information on the DC P3520. Intel briefed us on the DC P3320 during its Broadwell disclosure. However, it kept information about the DC P3520 to itself. Don't worry, though. We have the scoop on all of Intel's new SSDs, including dual-port NVMe-based offerings, on page three.

First, let's take a closer look at the Broadwell-EP microarchitecture.

Broadwell-EP Architecture

The Broadwell-EP line-up is based on three different die configurations with modular designs. The HCC die measures 18.1x25.2mm and comprises ~7.2 billion transistors. The architecture itself still employs two full rings per HCC die, but now it's symmetrical. In Haswell-EP, the ring on the right serviced two additional cores, creating asymmetry.

Here, Intel connects both bidirectional rings to 12 cores each, and it disables an equal number of cores per ring to create SKUs with fewer cores. As an example, the flagship 22-core Xeon E5-2699 v4 has 11 active cores per ring. As you work your way down the stack, two cores at a time are turned off, one from each side, along with their corresponding slices of last-level cache. That's how Intel creates models with less L3, too.

Each active core is associated with 2.5MB of LLC cache that is shared across its ring, and any core can address any part of the cache. The advantage of two distinct rings is more efficient scheduling; everything that happens on one ring is independent and occurs without any interference from the other ring. Routing ring traffic intelligently, and in the correct direction, is naturally quite important; a transaction on the ring can take up to 12 cycles (depending on how far it has to travel). There's intelligence built in to address this. Without it, if a core needed information in cache to the "south" of it and the traffic went north, that request would have to make a complete loop. Instead, the scheduler correctly routes traffic south, yielding faster access to data in the cache.

Balancing a workload between two rings also reduces the number of cycles that would be required to navigate one larger ring. The only caveat is that routing traffic between rings requires a trip across the buffered switches connecting them at the top and bottom, which incurs a (roughly) five-cycle delay. Each ring has access to its own memory controller (bottom), but only the ring on the left has access to the QPI links and PCIe lanes (top).

The MCC die measures 16.2x18.9mm and has ~4.7 billion transistors, while the LLC die measures 16.2x15.2mm and employs ~3.2 billion transistors.

Intel drops the number of cores per ring from 12 to 10 on the MCC and LCC configurations, but continues to employ a bidirectional ring structure. The MCC's partially severed ring even gets an additional memory controller. Then Intel does remove the second ring's last traces for the Low Core Count (LCC) die, eliminating it and the other memory controller. This also gets rid of any reason to have the buffered switches, which connected the two rings on the larger dies.

LCC-based models can still address four DDR4 memory channels through the single controller, illustrated by the four arrows emanating from that piece of logic. This results in a small loss of throughput, since there isn't a second memory scheduler to help service transactions. But Intel doesn't quantify the extent of the performance impact.

Performance Boosting Technologies

Broadwell-based CPUs boast a roughly 5.5% IPC boost compared to Haswell. The most notable improvements affect floating-point instruction performance, and include a reduction in Vector FP multiply latency from five cycles to three, improvements to the Radix-1024 divider, split scalar divider and hardware assist for vector gather operations (60 percent fewer).

Other compelling additions include virtualization-centric features like posted interrupts, which reduce VM enter/exit latency by batching the interrupts, and page modification logging, minimizing the overhead of VM-based fault tolerance through rapid checkpointing.

Intel also employs Transactional Synchronization Extensions (TSX) to boost performance, and its new Hardware Controlled Power Management purportedly cuts power consumption. We'll put that claim to the test on page eight. 

Orchestration And Security Features

Intel's Resource Director Technology provides enhanced telemetry data, which allows administrators to automate provisioning and increase resource utilization. This includes Cache Allocation Technology, Code and Data Prioritization (CDP), Memory Bandwidth Motioning (MBM) and enhanced Cache Monitoring Technology (CMT).

You also get a spate of enhanced security features, including faster data encryption and decryption, network security and trusted compute pools through Crypto Speedup (ADOX/ADCX), a new random seed generator (RDSEED), Supervisor Mode Access Prevention (SMNAP) and Virtualization Exception (#VE) technology.

Models And Pricing

Intel Test Platforms And How We Test

How We Test

We tested the Broadwell-EP-based Xeon E5-2697 v4 on both an Intel Software Development Platform server and the production-ready Supermicro SYS-1028U-TN10RT+. We tested the Haswell-EP-based Xeon E5-2699 v3 and E5-2643 v3 on the Intel Software Development Platform. And we tested the Ivy Bridge-based (v2) CPUs in Intel's Server System R2208GZ4GC. 

We benchmark with the open source Linux-Bench script, which is available on Linux-Bench.com and GitHub. ServeTheHome and others in the open source community maintain it. The suite runs from an Ubuntu 14.04 LiveCD either on local storage or through a KVM-over-IP connection. The script installs dependencies and runs several well-known independent open source benchmarks that characterize CPU performance.

Intel "Wildcat Pass" S2G3SY1Q Server

Intel sent us a pre-production Grantley-R EP S2G3SY1Q (Wildcat Pass) Broadwell Qualification system for our tests. The 2U test bed came with two Xeon E5-2697 v4 CPUs with 18 Hyper-Threaded cores and 45MB of shared cache apiece (that's 2.5MB of last-level cache per core). These beefy CPUs feature a non-AVX base clock of 2.3GHz and a maximum Turbo Boost frequency of 3.6GHz. The E5-2697 v4 offers a base clock of 2.0GHz and a Turbo Boost ceiling of 2.6GHz under AVX-optimized workloads.

The test platform features Intel's C610 chipset family and includes eight 32GB SK hynix DDR4-2400 DIMMs (HMA84GL7AMR4N-UH). Two riser cards enable PCIe connectivity, but weren't used in our testing.

Intel provides this server for use as a software development platform; it's not designed for use in a production environment. As such, it lacks some of the features that facilitate redundancy, such as dual PSUs. One of the PSU bays is covered, but the other houses a single 900W power supply.

Intel Server System R2208GZ4GC

An Intel R2208GZ4GC functions as one of the workhorses in our enterprise storage lab, and it has the scratches, bumps and bruises to prove it. At its heart, the server features the S2600GZ motherboard (C602 chipset) housed in a production-class chassis with the requisite redundant and hot-swappable fans, along with dual hot-swappable 750W power supplies.

Two riser cards have seen more than their fair share of RAID controllers, HBAs and PCIe SSDs, but again, they weren't required for today's benchmarks. We installed 64GB of Kingston DDR3-1600 memory in 8GB modules. Other notable platform features include quad 1GbE connections and an RMM4 module for remote management.

Supermicro And NVMe RAID Testing: 3 Million IOPS And 21 GB/s

Supermicro SuperServer SYS-1028U-TN10RT+

We already had the slender 1U Supermicro SuperServer SYS-1028U-TN10RT+ in our enterprise storage lab, so after a quick BIOS update we slapped in the Xeon E5-2679 v4 CPUs and went to work.

Don't let its diminutive stature fool you; the SuperServer is a little powerhouse. It features 10 hot-swap NVMe bays that we populated with ten 400GB 2.5" Intel DC P3700 SSDs. Normally, 16 counter-rotating fans provide copious airflow over 24 Samsung 16GB DDR4-2133 ECC DIMMs, but we removed 16 of them to level the playing field. Even with 384GB of DDR4 at our disposal, we are not populating the server to capacity; it supports an amazing 1.5TB of DDR4 ECC LRDIMMs. 

The NVMe SSDs mate to the PCIe backplane, which Supermicro routes to a pair of switchless expansion cards in the rear. These cards provide direct PCIe 3.0 x4 connections to the CPU, enabling incredible performance. The back of the chassis features two 10GbE ports and one management port, along with dual Titanium-rated 1000W redundant power supplies.

Need a slice of parallel processing in the storage sandwich? You also get three auxiliary eight-pin power connectors, allowing the server to accommodate either two full-height full-length graphics cards, or one double-width board and a low-profile offering.

We configured the 10 Intel NVMe SSDs into a RAID 0 volume (mdadm) in CentOS 7, demonstrating how the E5 2697 v4's 36 threads come into play under heavy I/O operations. The colored spaghetti strings in the picture above indicate the load of each core during a 32-thread 4KB random read workload.

Incredibly, we're pushing just over 3 million IOPS during this capture with an aggregate CPU utilization of 90.9 percent. To put that into perspective, you're seeing nearly 12 GB/s of random throughput. Now you see why very fast multi-core CPUs are so important to extracting maximum performance from NVMe-based SSDs in servers.

These results are hard to beat; we recently took 64 of the fastest PCIe SSDs out for a spin in 64 PCIe SSDs, 120TB Of Flash And One Stop Systems: The FSA 200 Review. The FAS 200 topped out with 3,055,735 IOPS in a system that monopolizes 5U of rack space. Meanwhile, the 1U Supermicro server musters even more random read performance thanks to its switchless design. We reach 814,351 IOPS during the 4KB random write workload, which is excellent scaling considering that Intel specs each drive at 75,000 random write IOPS.

The SuperServer continues to impress with a beastly 21 GB/s of sequential read throughput (yes, gigabytes), easily surpassing the 13 GB/s we achieved with the FAS 200. 

It is hard not to like this much power packed into such a small footprint; it's the personification of both performance and storage density. Now that we have seen what Intel's current-gen SSDs can do, let's take a look at the next generation.

Intel's 3D NAND SSD Debut: DC P3520/P3320 And DC3700/3600

DC P3520 And DC P3320 Series SSDs

Intel is simultaneously launching four datacenter SSDs, which are particularly interesting because the DC P3520 and DC P3320 both employ the company's long-awaited 3D NAND. The jointly developed IMFT flash sets a new standard for density with 32 layers and 256Gb per MLC die and an impressive 384Gb per die of 3D TLC.

The new NAND enables incredible density. We'll see up to 512GB for a 16-die MLC package and up to 768GB in a 16-die TLC package. IMFT 3D NAND employs floating-gate technology, which is quite a bit different than what other NAND fabs are doing with their Charge Trap Flash (CTF). Intel has a long history with floating-gate transistors, and the company believes it can extract more performance and endurance compared to competing CTF designs. IMFT NAND also puts most of the CMOS under the array, which boosts efficiency and density.

We were able to snap a few pictures of the DC P3520 at Intel's Broadwell-EP press briefing. As you can see, the SSD is physically almost identical to the company's existing DC P3700 SSDs.

The DC P3320 comes in capacities as large as 2TB, matching its counterparts based on planar NAND. This is surprising considering the higher density possible with 3D NAND. But Intel indicated that the new design features the same controller as the previous-gen planar NAND-based products. That controller can only address up to 2TB due to DRAM management limitations.

Intel sells the DC P3320 in both 2.5" and add-in card form factors with 450GB (2.5"-only), 1.2TB and 2TB of flash. The company rates its DC P3320 for up to 365,000 random read and 22,000 random write IOPS. Those write numbers are similar to the low-endurance DC P3500 (23,000 IOPS), but the random reads are not nearly as fast as the DC P3500's 430,000 IOPS. The DC P3320 offers up to 1600 MB/s sequential reads and 1400 MB/s of sequential writes, which also lags its NVMe-based predecessors.

Intel attributes the lower performance to a loss of parallelism due to higher-capacity 3D NAND dies. Future models will sport an improved controller that manages more DRAM, thereby accommodating more capacity and parallelism, which ultimately improves performance. Intel doesn't get specific, but indicates that the DC P3320 will feature a lower price than the DC P3500 and a similar write endurance rating (0.3 DWPD for five years).

Intel continues its focus on performance consistency, touting the end-to-end data path protection that provides a resilience to silent data corruption. Intel is one of the few storage vendors confident enough to share field reliability data, and it continues to maintain an Annual Failure Rate (AFR) below 0.44%.

Intel also mentioned the new DC P3520 during its briefing. However, the company did not share any performance details. We did uncover a document last August that accurately foretold the DC P3608's release though, and it included information on the DC P3520. The slide above is a bit dated at this point; its listed availability was originally anticipated in Q4 2015. But other information in the same document proved true, so we are fairly confident that it still reflects the projected specifications. At the very least, Intel hinted that the DC P3520 will be a performance-oriented model, which agrees with the numbers listed above.

Intel DC D3700 And DC D3600 Series SSDs

Intel also announced dual-port active-active NVMe-based SSDs at the event, which closely resemble the 2.5" DC P3700 NVMe SSD we recently tested and the 2.5" DC P3320 pictured above.

SAS-attached SSDs still enjoy the advantage of two ports, which provide an active-active connection for High Availability (HA) features like multi-path and fail-over. These are extremely important in mission-critical applications, and NVMe's original lack of dual-port functionality hampered its adoption in many of the long-held SAS-based SSD bastions, such as all-flash arrays. In fact, we dinged the 2.5" Intel DC P3700 NVMe SSD for that exact reason in our review earlier this month.

The NVM Express consortium added dual-port support to later revisions of the standard, and Intel's DC D3700 and DC D3600 SSDs provide two PCIe 3.0 x2 connections into compatible backplanes. Although this allows multiple hosts to access the SSDs, it also cuts performance in half, from a top speed of 470,000 random 4KB read and 95,000 write IOPS, and 2100 MB/s sequential read 1500 MB/s sequential write throughput.

Intel's new NVMe SSDs are only available in a 2.5" form factor, and they require special backplanes to take advantage of the dual-port functionality. You'll only find the DC D3700 at 800GB and 1.6TB capacity points, though you get 10 DWPD of endurance. The DC D3600 is available in 1 and 2TB capacities with a 3 DWPD rating.

Features supported by the drives are similar to what you get from SAS-based SSDs, including reservations, scatter/gather lists, multiple namespaces and in-controller memory buffers. Intel indicates that the mulitple namespace capability runs in hardware, which is better than SAS' software-based approach, increasing efficiency.

Benchmarks

Unix Bench 5.1.3

Unix Bench is an open source GPLv2-licensed test suite that originated in 1983 at Mondash University. Byte Magazine then updated and expanded the test suite. It includes nine tests that quantify various aspects of system performance, but we focus on the Dhrystone and Whetstone metrics. The former focuses on string handling; there are no floating-point operations. Whetstone measures the speed and efficiency of floating-point operations typically employed in scientific applications.

Both Xeon E5-2697 v4-based systems take a tremendous lead over the competition in our multi-threaded Whetstone test, which is impressive because the -2699 v3 features an identical core count, cache and clock rate. Apparently a big boost in IPC throughput, among other optimizations, delivers a strong performance improvement.

Conversely, the E5-2697 v4 comes crashing down to the bottom of the single-threaded Whetstone test. Both v4-powered servers muster more performance in the Dhrystone test, but still fall behind the Xeon E5-2643 v3. This is not entirely surprising; the eight-core -2643 v3 sports a higher 3.2GHz base frequency and a slightly better 3.7GHz max Turbo Core clock rate.

The Xeon E5-2643 v3's lower score in the single-threaded Whetstone test is unexpected. These results come from an average of three runs, but we retested the -2643 v3 extensively to ensure its outcome was repeatable.

NAMD Scalable Molecular Dynamics

NAMD is a parallel molecular dynamics code that performs high-performance simulations of large biomolecular systems. It's designed to test the boundaries of parallel computing by scaling up to 500,000 cores. The Theoretical and Computational Biophysics Group at the University of Illinois at Urbana-Champaign developed and maintains the project, which is the recipient of several prestigious awards.

The Xeon E5-2697 v4 easily finishes with a comfortable lead over competing CPUs in this parallel workload. The previous-gen -2643 v3 lands in last place, which aligns with our expectations due to its lower core count in comparison to the other contenders.

NAS Parallel Benchmarks

The NAS Parallel Benchmarks (NPB) are a set of programs developed by the NASA Advanced Supercomputing Division to evaluate the performance of parallel supercomputers. The test suite characterizes Computational Fluid Dynamics (CFD) applications, and NASA designed it to measure performance from smaller CFD applications up to "embarrassingly parallel" operations. 

The two Xeon E5-2697 v4 configurations are closely matched in the 64x64x64 BT test, but the -2699 v3 noses in between them with an impressive score as well.

The Broadwell-EP-based processors in Intel's development platform perform as we'd expect them to, while the same Xeons in Supermicro's enclosure lose quite a bit of steam. This suggests that the test's more complex portion is influenced to some extent by system memory performance. After all, the Intel chassis is rocking 2400 MT/s DDR4, while the Supermicro platform uses more pedestrian 2100 MT/s RAM. 

The M/ops per thread results indicate the performance of single cores during the test; they align with our expectations.

Redis

REmote DIctionary Server (Redis) is a popular in-memory data structure store that is used as a database, cache and message broker. It's dependent on both CPU performance and memory bandwidth. Redis is open source (BSD-licensed) software.  

The Xeon E5-2697 v4s in Intel's chassis take the lead thanks to higher memory bandwidth than the Supermicro-based platform. We also tested the -2643 v3 and -2699 v3 processors in the Intel server, which explains their stellar performance in this benchmark. Clock rate clearly comes into play as well (the -2643 musters impressive performance). Meanwhile, the second-gen E5s based on Ivy Bridge languish at the bottom of the chart due, in part, to their DDR3 memory subsystem.

Open SSL

SSL (Secure Sockets Layer) is the go-to technology for securing encrypted links between a server and client. It's an important protocol for every level of the Web in today's security-conscious climate.

According to Intel, encryption/decryption performance was one of the Broadwell team's targets during the design phase, and the Xeon E5-2697 v4 demonstrates the results of those efforts. As you can see, our OpenSSL test utilizes multiple cores well.

Additional Benchmarks

Sysbench CPU

Sysbench is a widely used suite that characterizes CPU, memory, file I/O, mutex performance and MySQL performance. We focus on the CPU test, which measures the amount of time required to verify prime numbers in both single- and multi-threaded workloads.

Both Xeon E5-2697 v4-powered servers provide impressive performance, beating most of the competition in the threaded component of this test. But Intel's -2699 v3 spoils the party by reporting a slightly faster result. The v4s fire back, overtaking the Xeon E5-2699 v3 in single-threaded performance, even besting the E5-2643 v3 as well.

Stream

Stream is a relatively simple test developed by Dr. John D. McCalpin. It measures the sustainable memory throughput of a given system in MB/s.

The Intel validation server employs speedy SK hynix DDR4-2400. As such, it leads our bandwidth results. The Xeon E5-2699 v3 employs dual memory controllers, which apparently are not as efficient as what Broadwell-EP brings to bear. They trail, even though they are tested on the same Intel motherboard.

We also tested the Xeon E5-2643 v3 on this board, and it trails significantly. The eight-core die only employs one memory controller, which just can't keep up.

C-Ray 1.1

C-Ray is a raytracing benchmark designed to reside entirely inside of the CPU caches, thus eliminating RAM and disk I/O overhead during the measurement window. The test focuses purely on floating-point performance during rendering and runs on multiple threads.

The -2697 v4s and -2699 v3 are locked in a three-way tie for the lead, while the second-generation E5s fall to the rear. Because this test scales well across multiple threads, the eight-core Xeon E5-2643 v3 is quickly outclassed, despite high clock rates.

7-Zip

7-Zip is open source software that measures compression and decompression performance, which can be a key capability for storage and networking applications.

The only surprise during this test is just how well the 18-core v4s fare compared to the Xeon E5-2699 v3. Meanwhile, all of the second-gen E5s we tested fare dismally, illustrating the benefits of a more modern platform. Most enterprises will be upgrading from Sandy/Ivy Bridge to Broadwell, and this benchmark does a good job of highlighting the type of performance boost to expect.

HardInfo

HardInfo provides granular system information, and includes a suite of six benchmarks that measure CPU performance. It's easily accessible and comes as a standard component in many Ubuntu desktop systems. We include these tests because they allow our Linux brethren to easily run comparative benchmarks.

The v4s lead in FPU raytracing, but fall behind their predecessors in the Fibonacci test. This generational face-off gets a little more heated during the FPU FTT tests, and the Xeon E5-2687 v4 in Intel's development platform dominates the Cryptohash test.

Power Consumption

There is no way to understate the impact of power consumption and efficiency on the modern datacenter. Power use is the killer of all things bottom-line. Datacenters worldwide consumed an estimated 416.2 terawatts (416.2 trillion watts) of electricity in 2014, which is more than 182 entire countries (out of 192 globally). Even more alarming, datacenter power use is quadrupling every two years, and a recent Japanese study concluded that at today's rate, its datacenters will consume all of the country's electricity output by 2030.

The pressure to reduce power consumption is an overbearing burden, but at the same time, the demands for more processing power are increasing every year. Deploying more efficient CPUs simultaneously increases performance per watt, frees up floor space and relaxes cooling requirements. All three of those variables function as multipliers that can either reduce or inflate datacenter expenditures dramatically.

Intel took several important steps to reduce power consumption with its previous-generation Haswell-EP products, including the addition of an on-package power delivery system. It also added per-core P-state control and moved to DDR4 memory, which helped reduce platform power use, further improving efficiency.

The Xeon E5-2600 v4 family offers these same advantages, but also brings about the introduction of Hardware Controlled Power Management (HWPM). Under normal circumstances, most servers accept hints from the operating system that indicate when it is appropriate to adjust the power state. Unfortunately, this process is slow in relation to how fast a CPU can make decisions internally, and not all software developers are utilizing the feature.

HWPM turns power management over to the CPU, offering up to four power profiles that optimize the server for each use-case, which you can specify in the BIOS. The CPU then adjust power state settings dynamically, depending on the chosen profile. Not only is this process faster, but it's also compatible with all operating systems (the CPU simply ignores the OS hints). Intel is in the early stages of HWPM development, but expects the feature to evolve rapidly.

Linux-Bench Power Consumption

We measured power consumption during the Linux-Bench script, which provides a nice comparison point for each system as the test progresses. The first slide provides a view of the test servers in one image. Then, we provide two additional slides that paint a clearer picture.

The Supermicro platform is unsurprisingly more efficient than Intel's software development platform. After all, it's a production-class system with 80 PLUS Titanium-rated PSUs.

The Xeon E5-2699 v3 CPUs consume more power than our Broadwell-EP samples in the same Intel development platform, which speaks volumes given their similar core count. The -2643 v3 consumes less power by virtue of its lower core count. This is an important consideration; it is not wise to provision excess CPU resources that exceed the requirements of the workload. It's better to pick the right CPU for your application. 

Power Load And Idle

We measured peak power consumption during a Linpack run to characterize each platform's use. Again, Supermicro's configuration is more efficient than Intel's in both maximum and idle consumption. We also recorded the watt-hours consumed during the entire Linux-Bench script, and found the Supermicro platform offering a more refined efficiency story. The second-gen Xeon E5s do use less power, but they're also a lot slower.

Conclusion

Intel's switch to the process, architecture and optimization cadence is a clear indicator that the heady days of Moore's Law is drawing to a close. Over the last few decades, Intel picked the low-hanging fruit from the transistor tree as it moved forward at an incredible pace, but now we're finding each processor generation offering smaller and smaller performance improvements.

The majority of datacenter and enterprise customers are locked into three- or five-year refresh cycles due to maintenance contracts, and as those contracts expire, they upgrade to the newest platforms. This means that the majority of Broadwell-EP customers will be migrating from either Sandy Bridge or Ivy Bridge, and not replacing Haswell-EP-based processors with newer Broadwell-EP models.

As a result, they'll reap the benefits of not only Intel's Broadwell-EP design, but also the mature C610-series chipsets, which continue to find use with the fourth-generation Xeon E5s (updated with fresh firmware, of course). That platform controller hub family merely serves to support a well-endowed host processor. The PCH has limited PCIe 2.0, some built-in USB 3.0 and quite a bit of SATA connectivity. Really, it's the Xeon's big PCIe 3.0 controller, fast QPI links and DDR4 memory controllers (now able to accommodate 2400 MT/s modules) that do the heavy lifting.

The CPU's key advances include a top-end model with four more physical cores, a roughly 5.5 percent improvement in IPC throughput and a last-level cache increase from 45 to 55MB. Our benchmarks show the Xeon E5-2600 v4 serving up nice performance increases through most of the suite. Numerous platform-level improvements offer massive performance advantages to anyone migrating from old Sandy Bridge and Ivy Bridge servers. On the flip side, there's probably not enough reason for most businesses to ditch Haswell-EP in favor of Broadwell-EP, unless they really need a specific new feature.

Speaking of new features, they appear to be an increasing focus at Intel as the glory days of massive performance gains fade into history. In lieu of big speed-ups, Broadwell-EP includes some extras designed to enhance performance and usability in a variety of applications. VM-centric optimizations should enjoy considerable use in the wild, and the new orchestration tools allow administrators to monitor, manage and optimize with a heretofore-unseen level of granularity. Increased performance and manageability in security-related tasks through faster encryption/decryption, a new random seed generator, SMNAP and Crypto Speedup will come in handy as well.

Intel is wisely using its Broadwell-EP launch as a springboard to introduce a number of high-powered SSDs, too. The meteoric rise of solid-state storage in the datacenter is making it easier to fully utilize potent multi-core CPUs, and matching speedy NVMe-based drives up to the latest in processor technology proves key in bleeding-edge applications, as evidenced by our NVMe RAID tests.

Many enthusiasts are looking to Skylake as the next truly revolutionary advance in processor technology, and there are predictions that it will support 3D XPoint additives and on-die Omni-Path adapters. Sure, Skylake is exciting. But the 14nm node might have more to offer in the near term as well.

Intel recently displayed Arria 10 FPGAs and what appeared to be a Broadwell die on the same package at the Open Compute Summit. It announced that products with this design will ship in 2016. The company's recent Altera purchase looks like it's bearing fruit already, and we may see an interesting marriage of CPU and FPGA on the same die during the transition to process, architecture and optimization.

In the meantime, Intel's Xeon E5-2600 v4 family provides an attractive upgrade path that offers gains in performance, cores, cache and power consumption that will satisfy the vast majority of users, and in particular, those migrating from Ivy and Sandy Bridge. 

MORE: Best CPUs
MORE: Intel & AMD Processor Hierarchy
MORE: All CPU Content

Paul Alcorn is a Contributing Editor for Tom's IT Pro and Tom's Hardware, covering Storage. Follow him on Twitter and on Google+. Follow us on Facebook, Google+, RSS, Twitter and YouTube.