Intel talks Meteor Lake AI and efficiency: Head of performance marketing lab is 'most excited' about battery life improvements

(Image credit: Intel)

Intel's Core Ultra chips (formerly Meteor Lake) launched on December 14th, and are already appearing in laptops you can buy. At Intel's official launch here in New York City, I sat down with Dan Rogers, the head of Intel's performance marketing lab in its Client Computing Group.

Rogers, the former product manager for Alder Lake and Raptor Lake, has a unique job as Intel pushes out its first chips based around AI: figuring out how to tell the story of what these can do, and also measuring their performance for Intel's marketing to the general public (Intel is in the PC business, after all).

Because right now, while Intel (and, let's be frank - AMD, Qualcomm, and Apple) have systems with neural processing units, or NPUs, lots of potential customers don't quite know what they could do with them, or why it would matter. I asked him about how people are supposed to know what an AI PC can do, when they can expect software that will affect their lives, and also learned more about how Intel is benchmarking AI applications internally.

Selling the AI PC

Intel's debut mobile processor with an on-board NPU is coming as the company is making a major brand name change. Intel Core i is old hat. Intel Core Ultra is your AI-equipped chip. Rogers suggested to me that the name change is part of the "multiple levels of education" that Intel will go through to inform customers about what they're getting.

"All of our processors code named 'Meteor Lake' will be Core Ultra and all Core Ultra processors will be Meteor Lake," Rogers said. "So, it's a fairly simple method of communicating to the end user, right, especially at the point of sale and retail and so forth that this is a PC that has built-in AI."

How people will actually use an NPU or other AI support, however, is a bit more nuanced. Rogers suggested to me that while AI has been foreign to a lot of people until now, he expects that won't be the case going forward as Zoom, Adobe, and other name-brand PC software start shipping features. After all, while you can download an OpenVINO plugin for GIMP or Audacity, but let's be real — most people won't do that.

"Definitely there are more open source community tech enthusiast-oriented usages for sure, early adopters and folks that want to really dig in," he said. "But, you know, also for just the general user, much of what they do today will now be made better through AI.

Otherwise, Rogers said that Intel is relying on three major points to convince people to make the leap. The first point, of course, is the NPU. The second is the GPU, which is seeing Arc-branded graphics for the first time. The third is battery life — Intel is promising a way more efficient machine, especially with certain tasks from the CPU and GPU offloaded to the NPU. (I'm very curious to test this.)

Is Intel still a CPU company in 2024?

So Intel has a three-pronged message: NPU, GPU, battery. Huh.

Perhaps I was being facetious, but I couldn't help but notice one thing as Intel introduced the new processors: a lack of emphasis on the CPUs. The CPU felt like it took a backseat. Is Intel, I asked, still a CPU company?

"[The CPU is] extremely important to us… We still believe we have a really strong position in the core technology, but absolutely, Meteor Lake kind of builds on what we were able to do with

the Core and performance hybrid architecture and Alder and Raptor for performance," Rogers said. "Now we have a third level of hybrid architecture for power efficiency, and then we're working on GPU and NPU as well.

"So, you know, as the kind of former Alder and Raptor guy, for me, it's a cool product because,

you know, it sort of compliments those advances that we had in the prior gen now working across all the other vectors of compute and efficiency."

For now, that's starting in Intel's mobile chips. It sure sounds like some of Meteor Lake's tech is coming to desktop soon, though. Rogers said that Intel will "share more later in the year" regarding moving the architecture into other computing segments.

The chicken and the egg

So you have an AI PC. Now what? 

That's the "chicken and the egg" problem of NPUs. They're out there, and now developers need to make software work with them.

I gave Rogers the case of my brother, who doesn't have a very complicated use case for work: he does emails; he does spreadsheets; he's got documents. He's still on Windows 10. Would he even notice?

"For sure, I would guess your brother is watching YouTube and Netflix," Rogers suggested. (He's right on YouTube. He doesn't have Netflix.) "Quite likely, he's making a family call or a business call with Zoom or Teams or WebEx, in these popular applications."

And these, Rogers tells me, are "much better" on Meteor Lake laptops. Video effects in Zoom, for instance, will be offloaded from the GPU to the NPU, helping to save power, leading to longer battery life. That, along with the "low-power island", including efficiency cores directly incorporated into the SoC tile, should save power.

"That-low power island is a really important technology for us, and we spent a ton of time in software tuning that to make that seamless, so that your brother doesn't necessarily need to know how all the bits are actually working there, but in the end, web browsing and video playback both locally streaming, just longer battery life, just better battery life, right? And that's all with similar performance to what we had in Raptor Lake."

What I got from this was that to start, the best AI features might not be ones you actually notice. Do you care if you know that your applications are using AI or an NPU as long as you're getting some sort of tangible benefit? Maybe not!

But Intel is promising that over 100 software companies are working with it to put out features that are optimized for the NPU. So when will we see those features come that really push people to new laptops with new CPUs?

Well, Intel has demos of Llama 2-based applications — I saw one at Intel's showcase — and Rogers expects to see LLMs as one major way app developers will use the NPU locally.

"My expectation is we're going to see not just one, but tens and hundreds of these from various ISVs, preloaded with various OEMs," he said. "And there's a ton of energy going on on this and people looking to run these both on the GPU and the NPU."

But Intel suggests it has another benefit on its side: scale. Rogers said that a "giant developer ramp" is required to get coders working on these types of projects, and that "in many cases, we have developers coming to us" for guidance with tools and frameworks.

How do you benchmark PCs on AI?

I test hardware for a living. So I had to ask, how did Intel decide how to test Meteor Lake on AI? The claims are bold, but there are only so many workloads to pick from right now.

"We're talking about the PC here right? So of course we want to benchmark it, right? That's just how we handle understanding PC technologies."

Rogers said his team went in looking to explain how the NPU compares to say, running a workload on the GPU. But it ends up that much of the benchmark, when it comes to AI applications, is really about something else: software.

"What we realized really quickly is we're not really benchmarking the hardware — we're actually benchmarking the code paths," he said. "We're benchmarking the software, which is really interesting, right? Like that's very different than what we see today in the CPU and the GPU."

That, Rogers said, is why Intel shared a chart showing that Meteor Lake runs all sorts of AI workloads, and does so "performantly." (AMD and Qualcomm's offerings were also listed, with lesser degrees of success.)

(Image credit: Intel)

And benchmarking AI tools isn't cut and dry in other ways, Rogers said. "What's happening right now is okay, how big of a model can I run? And that's a question of computing power, how do I quantize that model, how do I minimize not just the compute but also the memory footprint and the memory bandwidth requirements which are enormous for many of these."

This isn't a hardware story, not entirely, Rogers said. You can come back to an improved model in weeks, he said, and see massive gains. This is also about software — or at least software optimization.

Goodbye, P-Series! Hello, H-Series?

Intel has simplified its naming scheme a bit with Meteor Lake, removing the "P series," which has only been around for a couple of generations, and wrapping that into the long-existing H-series. Now, there's just H and U.

H series is the larger chip, with a 6+8 die (6 P-cores, 8 E-cores, though there are processors with 4 P-cores), while the U series is designed for 2+8. 

Perhaps the biggest difference is that the H series is running at 28W, which the P series ran at. On Raptor Lake, the H chips ran at 45W. When the Intel Core Ultra 9 185H launches in early 2024, that will be a 45W part. So Intel still has a range of TDPs. It's simpler in naming, but not inherently in power.

Swipe to scroll horizontally
Header Cell - Column 0 Intel Core Ultra 9 185H*Intel Core Ultra 7 165HIntel Core Ultra 7 155HIntel Core Ultra 5 135HIntel Core Ultra 5 125H
E-Cores8 (2 LP)8 (2 LP)8 (2 LP)8 (2 LP)7 (2 LP)
Smart Cache (LLC)24M24M24M18M18M
Max turbo frequency (GHz, P-cores/E-cores)5.1 / 3.85.0 / 3.84.8/3.84.6/3.64.5 / 3.6
Integrated GPUIntel Arc GPU, 2.35 GHz Max, 8 Xe coresIntel Arc GPU, 2.3 GHz Max, 8 Xe coresIntel Arc GPU, 2.25 GHz Max, 8 Xe coresIntel Arc GPU, 2.2 GHz Max, 7 Xe coresIntel Arc GPU, 2.2 GHz Max, 7 Xe cores
Neural ProcessorIntel AI Boost, 2x Gen 3Intel AI Boost, 2x Gen 3Intel AI Boost, 2x Gen 3Intel AI Boost, 2x Gen 3Intel AI Boost, 2x Gen 3
Max Memory SpeedDDR5-5600, LPDDR/x-7467DDR5-5600, LPDDR/x-7467DDR5-5600, LPDDR/x-7467DDR5-5600, LPDDR/x-7467DDR5-5600, LPDDR/x-7467
Max Memory Capacity64GB (LP5)/96GB (DDR5)64GB (LP5)/96GB (DDR5)64GB (LP5)/96GB (DDR5)64GB (LP5)/96GB (DDR5)64GB (LP5)/96GB (DDR5)
Base power (W)45W28W28W28W28W
Turbo power (W)115W64W, 115W64W, 115W64W, 115W64W, 115W

"In [the] Meteor Lake generation it will be largely what was our P series, so that thin and light performance — thin and light concept — packing a lot of power into an integrated graphics system," Rogers said.

But that means that people could, in theory, go into a store and think they're getting a different amount of power. How, I asked, will Intel make it clear what you're getting?

"Part of improving communication is simplifying, so we definitely got the feedback and we wanted to just simplify the number of choices," he said. So that's why we made it from a three-letter structure to a two-letter structure which we think is helpful."

Rogers said that the decision will really be made when a customer goes to a store and looks for what they want to buy. That messaging, he said, will come from the company that makes the laptop and what they offer in their lineup.

"And then as a user, you're just able to kind of see, 'OK, what are my needs,' right? Do I want lighter, do I want a little bit more performance? So we have a letter that gives that choice, but really that's a choice that you're going to see in the exact laptops that you're interested in purchasing. So if you know that you're looking for a more powerful [PC], lots of those laptops generally will have H-series."

9W parts and "very different" form factors

Intel didn't have a low-wattage part available in 13th Gen mobile, which led to 12th Gen chips in foldables released this year. In Intel's list of chips coming in Q1 of 2024, there's the Core Ultra 7 164U and Core Ultra 5 134U, both of which are 9W processors. So what new form factors could Intel be trying to enable by going back to low-power parts?

"Without pre-announcing from my friends, colleagues, and partners I think you will see some, notable, very different form factors for Core Ultra, not necessarily in the PC space," Rogers said, pausing.

"It's a PC, but in a different category, let's say," he resumed to finish his thought. I couldn't help but think that every major gaming handheld so far has used AMD Ryzen processors. There's also Lenovo's rollable prototype from Mobile World Congress. At the very least, we could get more modern foldables.

But most Meteor Lake designs will be two-in-ones, clamshells, and other more traditional shapes, he said. "Most of the focus of our partners is actually driving more compute into thinner, lighter envelopes. Thermal technologies have gotten a lot better." Rogers suggested that the increased thermal capacity means you can put a 28W part in a system you used to put a 15W CPU into.

And because AI can be computationally demanding, Rogers predicted that you'll see more computers with the higher TDP choice (the 28W H-series) rather than the lower-power 15W U-series chips, though both will be available.

"At the end of the day, OEMs make the right decision based on their form factor, but we'd like to provide choice," he said. "And I think the biggest thing that I'm most excited about in Core Ultra, for me, is the battery life improvements." All of the data Intel showed at launch, he pointed out, was on the 28W part.

We'll see what Intel and laptop companies come up with over the next year. While a handful of devices from MSI, Lenovo, Dell, HP, Asus, Acer, Samsung, Gigabyte, and more launched this week, CES 2024 is right around the corner. And what would a new year be without a slew of new laptops? We'll see then.

Andrew E. Freedman is a senior editor at Tom's Hardware focusing on laptops, desktops and gaming. He also keeps up with the latest news. A lover of all things gaming and tech, his previous work has shown up in Tom's Guide, Laptop Mag, Kotaku, PCMag and Complex, among others. Follow him on Threads @FreedmanAE and Mastodon

  • kjfatl
    It looks like all of these parts come from the same mask set. During testing the parts that test good for low power are given the low power SKU. The parts remaining parts are given a high power SKU. At the end of the testing process fuses are blown to configure the part appropriately.
  • JamesJones44
    we're actually benchmarking the code paths
    That's what all benchmarking is. Whether it's benchmarking general compute, SMID instructions, FPS, etc. those are "code paths" and benchmarking a specific piece of software against various different hardware configurations. I'm not sure why Intel believes the NPU is any different. Running a ResNet or YOLO model on various NPUs to judge relative performance from one NPU to the next is no different than running a compression test on two different CPUs.
  • ThomasKinsley
    The last laptop I got is a decade old and the best battery life was maybe 4 hours? Now it's down to 2, and that's with a fuzzy 1366x768 screen. I might end up checking one of these out if the battery is good.
  • endocine
    thermals looked insane on those meteor lake laptop previews, is 106C really OK?
  • peachpuff
    Battery life and 115w turbo boost go hand in hand 👌
  • syadnom
    It's all even more confusing to the end user. Intel Core Ultra 9 185H? that's a mouthful and what does it mean? And the mix of P and E cores and how that isn't really reflected in the part number is a concern as well. Turbo at 115W? or even 65W?

    I feel like they are sort of trying to copy Apple with a big push into P & E core mixes, but they are still handing us >100W *laptop* CPUs.

    How exactly does this compete with Apple hardware first of all, and then we have upcoming arm64 windows machines that are maybe not quite as fast as Apple's M2 and M3, but they are actually respectable performance and in a fraction of the power envelope. ie, that base 28W is actually max and idle is under 10W.

    Not making this a 'mac' argument, just that Apple showed everyone what is possible and Qualcomm is fighting very very hard to make windows machines that do the same. Intel needs to do a lot better than 28W/65W/115W in a mobile chip or the only things you'll find intel logos on are high end gaming laptops.

    Is intel going to be relegated to high powered desktop and barely portable gaming laptops? These chips simply don't look like 2024's laptop parts to me.
  • TerryLaze
    peachpuff said:
    Battery life and 115w turbo boost go hand in hand 👌
    Being able to finish a job in less time does help with battery life, especially if the efficiency peak point is higher up the power curve, so if the cpu can use a good amount of power and that helps it to finish a job in far less time it will do so with less total power.

    Also why would you want to be artificially limited to a low power draw on a 8p core system with a decently big iGPU? I understand it for when you are on battery, but why would you want to be forced to use little power when you are on mains?
  • Colif
    Some interesting points raised in

    seems to be a soft launch since nothing out there to test yet.
  • bluvg
    Colif said:
    Some interesting points raised in

    seems to be a soft launch since nothing out there to test yet.
    MLID is a great source and he would likely deny AMD bias, but I think he's let his MTL and general Intel disappointment cause him to lose perspective. For example, at 8:33 in that video, he says MTL is has "gone backwards in efficiency," while simultaneously showing a screenshot of a MTL benchmark from Dave2D showing it's far more efficient than Raptor Lake, almost double the battery life in one case.

    Availability is going to take a bit of time (keep an eye on CES), though a few are available now. He also only occasionally points out that AMD Phoenix and Hawk Point are not that widespread in laptops, either (sadly).
  • bluvg
    a handful of devices from MSI, Lenovo, Dell, HP, Asus, Acer, Samsung, Gigabyte, and more launched this week

    Where? I didn't see anything from HP.