AMD claims Ryzen smashes Intel's Meteor Lake in AI benchmarks — up to 79% faster at half the power consumption
Local AI processing in Llama 2 and Mistral Instruct 7B seem much faster on AMD.
AMD has shared some insight into its "consumer AI performance" with Tom's Hardware, featuring a face-off between the AMD Ryzen 7 7840U (15W) and Intel Core Ultra 7 155H (28W). In the selected large language model (LLM) performance tests using Llama 2 and Mistral Instruct 7B, we see that the red team's solution is far quicker, uses less power, and is available cheaper. Though we expect chipmakers to cherry-pick benchmarks, these results show Intel's AI performance isn't even close to the local AI processing horsepower AMD demonstrates here.
For its test comparisons, AMD installed LM Studio on the competing laptops – a task that it says takes just a few minutes. Once LM Studio is installed, you can run various LLMs locally for utmost privacy, subscription fee-free, and without an internet connection. As we mentioned in the intro, Llama 2 and Mistral Instruct 7B were leveraged for various tasks. Specifically, AMD got the AIs to write a story multiple times, make a demo with a bouncing ball in Unity, and write a poem. For some background information, Llama 2 is a state-of-the-art LLM from Meta, and Mistral Instruct 7B is an LLM with 7.3 billion parameters developed by ex-Meta and DeepMind devs.
AMD chose performance metrics to showcase its Ryzen's prowess, including measurements of 'time to first token'—which charts the time from inputting a prompt and pressing enter to the first results showing up. Another critical performance measure is 'tokens per second,' where an LLM response is delivered line by line.
As a reminder of the properties of the latest Ryzen chips that align themselves to local AI acceleration, AMD says that three kinds of cores can be used in concert: NPU, RDNA 3 GPU, and Zen 4 CPU. In the demo we watched, an AMD exec said that AVX512 and VNNI acceleration built into the Zen 4 CPU cores were behind the winning results you see in the charts. That's interesting, as we expected to be told about the more powerful NPU in the Phoenix vs Meteor Lake processors.
AMD has published a blog about how to run LM Studio and various LLMs on your local machine. The tutorial guides you through getting started on either an 'AMD Ryzen AI PC' or a PC with a Radeon RX 7000 or newer GPU installed.
Intel is well aware of the relatively paltry AI performance delivered by the Meteor Lake family. The iconic CPU maker has already officially teased that both Arrow Lake and Lunar Lake will have three times more AI performance for GPU and NPU. Intel's next-gen laptop and desktop processors are due later this year.
Stay On the Cutting Edge: Get the Tom's Hardware Newsletter
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Mark Tyson is a news editor at Tom's Hardware. He enjoys covering the full breadth of PC tech; from business and semiconductor design to products approaching the edge of reason.
-
hotaru251 ignoring the % results (as always take 1st party w/ pinch of salt) the power efficiency is a solid point given in many places electricity prices are rising a lot.Reply -
waltc3 Intel seems to be really big on boasting what its CPUs will do at some time in the future--it's the present tense that always seems a problem for them, I've noticed...;)Reply -
Alvar "Miles" Udell I can completely believe it since the chosen AMD processor had specific instructions the Intel processor did not. However, why were the full details not shared, Mark Tyson? I see on one side "SEE ENDNOTE PHX-59", and that is not present in the article. It is also not present anywhere on AMD's website, and Microsoft Copilot cannot find any reference to PHX-59. Presumably it lists the exact systems which were tested and could easily explain the results. Also, it says that AMD shared information with TomsHardware and does not give a source.Reply
Seeing as how this information isn't reported in any other publication, either reputable or disreputable, this sounds like either a paid advertisement by AMD, or information unauthorized for release at this time. If it were a general or press release it would be across the net by now, and would carry source citations. -
hotaru251 Alvar Miles Udell said:I see on one side "SEE ENDNOTE PHX-59",
this is endnotes found from r/hardware in a thread about this exact article. (google search is helpful for this stuff if you know how to use it to filter out the trash results) -
Alvar "Miles" Udell
"Testing as of February 2023"hotaru251 said:
this is endnotes found from r/hardware in a thread about this exact article. (google search is helpful for this stuff if you know how to use it to filter out the trash results)
"Market price retrieved 3/4/2023"
"Ryzen 7 7840U 15w TDP"
Intel Core Ultra 7 155H released in Q4 2023.
Ryzen 7 7840U has a 15-30w TDP, default 28w, 30w when used in the laptop specified, which was reviewed by TomsHardware in December 2023 and cost $1239.99
The base Pavilion Plus 14 comes with a Ryzen 5 7540U processor (Radeon 740M Graphics), 16GB of LPDDR5 memory, a 512GB PCIe 3.0 SSD, a 14-inch 1920 x 1200 IPS display, and a Wi-Fi 6/Bluetooth 5.3 combo card for $849. Upgrading to a 120Hz 2560 x 1600 IPS display adds $80 to the price while opting for a 512GB PCIe 4.0 SSD adds $20.
Our review unit came with the Ryzen 7 7840U processor (Radeon 780M Graphics), 16GB of LPDDR5 memory, a 1TB PCIe 4.0 SSD, a 14-inch 2800 x 1800 OLED display, and a Wi-Fi 6E/Bluetooth 5.3 combo card for $1,239.
https://www.tomshardware.com/laptops/hp-pavilion-plus-14-oled-review
So if AMD just didn't adjust the date to 2024, they are comparing a $1240 laptop to a $1000 Acer laptop (current price on Acer's website), 24% higher price for 14% and 17% faster tokens per second.
https://store.acer.com/en-us/swift-go-14-laptop-sfg14-72t-71qf
That makes this "article" even more suspect that -none- of this was mentioned. I'd say this "article" should be pulled or, at the least, amended with this information so it doesn't look like a paid AMD advertisement. -
Sluggotg
Of Course AMD has NEVER done that. You know.. with the Bulldozer Processor...waltc3 said:Intel seems to be really big on boasting what its CPUs will do at some time in the future--it's the present tense that always seems a problem for them, I've noticed...;) -
Sam Bridges I claim I’m bigger than young Schwarzenegger. That doesn’t mean it’s true. 🤣Reply
Cheers.