Intel talks Meteor Lake AI and efficiency: Head of performance marketing lab is 'most excited' about battery life improvements

Intel's Core Ultra chips (formerly Meteor Lake) launched on December 14th, and are already appearing in laptops you can buy. At Intel's official launch here in New York City, I sat down with Dan Rogers, the head of Intel's performance marketing lab in its Client Computing Group.

Rogers, the former product manager for Alder Lake and Raptor Lake, has a unique job as Intel pushes out its first chips based around AI: figuring out how to tell the story of what these can do, and also measuring their performance for Intel's marketing to the general public (Intel is in the PC business, after all).

Because right now, while Intel (and, let's be frank - AMD, Qualcomm, and Apple) have systems with neural processing units, or NPUs, lots of potential customers don't quite know what they could do with them, or why it would matter. I asked him about how people are supposed to know what an AI PC can do, when they can expect software that will affect their lives, and also learned more about how Intel is benchmarking AI applications internally.

Selling the AI PC

Intel's debut mobile processor with an on-board NPU is coming as the company is making a major brand name change. Intel Core i is old hat. Intel Core Ultra is your AI-equipped chip. Rogers suggested to me that the name change is part of the "multiple levels of education" that Intel will go through to inform customers about what they're getting.

"All of our processors code named 'Meteor Lake' will be Core Ultra and all Core Ultra processors will be Meteor Lake," Rogers said. "So, it's a fairly simple method of communicating to the end user, right, especially at the point of sale and retail and so forth that this is a PC that has built-in AI."

How people will actually use an NPU or other AI support, however, is a bit more nuanced. Rogers suggested to me that while AI has been foreign to a lot of people until now, he expects that won't be the case going forward as Zoom, Adobe, and other name-brand PC software start shipping features. After all, while you can download an OpenVINO plugin for GIMP or Audacity, but let's be real — most people won't do that.

"Definitely there are more open source community tech enthusiast-oriented usages for sure, early adopters and folks that want to really dig in," he said. "But, you know, also for just the general user, much of what they do today will now be made better through AI.

Otherwise, Rogers said that Intel is relying on three major points to convince people to make the leap. The first point, of course, is the NPU. The second is the GPU, which is seeing Arc-branded graphics for the first time. The third is battery life — Intel is promising a way more efficient machine, especially with certain tasks from the CPU and GPU offloaded to the NPU. (I'm very curious to test this.)

Is Intel still a CPU company in 2024?

So Intel has a three-pronged message: NPU, GPU, battery. Huh.

Perhaps I was being facetious, but I couldn't help but notice one thing as Intel introduced the new processors: a lack of emphasis on the CPUs. The CPU felt like it took a backseat. Is Intel, I asked, still a CPU company?

"[The CPU is] extremely important to us… We still believe we have a really strong position in the core technology, but absolutely, Meteor Lake kind of builds on what we were able to do with

the Core and performance hybrid architecture and Alder and Raptor for performance," Rogers said. "Now we have a third level of hybrid architecture for power efficiency, and then we're working on GPU and NPU as well.

"So, you know, as the kind of former Alder and Raptor guy, for me, it's a cool product because,

you know, it sort of compliments those advances that we had in the prior gen now working across all the other vectors of compute and efficiency."

For now, that's starting in Intel's mobile chips. It sure sounds like some of Meteor Lake's tech is coming to desktop soon, though. Rogers said that Intel will "share more later in the year" regarding moving the architecture into other computing segments.

The chicken and the egg

So you have an AI PC. Now what? 

That's the "chicken and the egg" problem of NPUs. They're out there, and now developers need to make software work with them.

I gave Rogers the case of my brother, who doesn't have a very complicated use case for work: he does emails; he does spreadsheets; he's got documents. He's still on Windows 10. Would he even notice?

"For sure, I would guess your brother is watching YouTube and Netflix," Rogers suggested. (He's right on YouTube. He doesn't have Netflix.) "Quite likely, he's making a family call or a business call with Zoom or Teams or WebEx, in these popular applications."

And these, Rogers tells me, are "much better" on Meteor Lake laptops. Video effects in Zoom, for instance, will be offloaded from the GPU to the NPU, helping to save power, leading to longer battery life. That, along with the "low-power island", including efficiency cores directly incorporated into the SoC tile, should save power.

"That-low power island is a really important technology for us, and we spent a ton of time in software tuning that to make that seamless, so that your brother doesn't necessarily need to know how all the bits are actually working there, but in the end, web browsing and video playback both locally streaming, just longer battery life, just better battery life, right? And that's all with similar performance to what we had in Raptor Lake."

What I got from this was that to start, the best AI features might not be ones you actually notice. Do you care if you know that your applications are using AI or an NPU as long as you're getting some sort of tangible benefit? Maybe not!

But Intel is promising that over 100 software companies are working with it to put out features that are optimized for the NPU. So when will we see those features come that really push people to new laptops with new CPUs?

Well, Intel has demos of Llama 2-based applications — I saw one at Intel's showcase — and Rogers expects to see LLMs as one major way app developers will use the NPU locally.

"My expectation is we're going to see not just one, but tens and hundreds of these from various ISVs, preloaded with various OEMs," he said. "And there's a ton of energy going on on this and people looking to run these both on the GPU and the NPU."

But Intel suggests it has another benefit on its side: scale. Rogers said that a "giant developer ramp" is required to get coders working on these types of projects, and that "in many cases, we have developers coming to us" for guidance with tools and frameworks.

How do you benchmark PCs on AI?

I test hardware for a living. So I had to ask, how did Intel decide how to test Meteor Lake on AI? The claims are bold, but there are only so many workloads to pick from right now.

"We're talking about the PC here right? So of course we want to benchmark it, right? That's just how we handle understanding PC technologies."

Rogers said his team went in looking to explain how the NPU compares to say, running a workload on the GPU. But it ends up that much of the benchmark, when it comes to AI applications, is really about something else: software.

"What we realized really quickly is we're not really benchmarking the hardware — we're actually benchmarking the code paths," he said. "We're benchmarking the software, which is really interesting, right? Like that's very different than what we see today in the CPU and the GPU."

That, Rogers said, is why Intel shared a chart showing that Meteor Lake runs all sorts of AI workloads, and does so "performantly." (AMD and Qualcomm's offerings were also listed, with lesser degrees of success.)

And benchmarking AI tools isn't cut and dry in other ways, Rogers said. "What's happening right now is okay, how big of a model can I run? And that's a question of computing power, how do I quantize that model, how do I minimize not just the compute but also the memory footprint and the memory bandwidth requirements which are enormous for many of these."

This isn't a hardware story, not entirely, Rogers said. You can come back to an improved model in weeks, he said, and see massive gains. This is also about software — or at least software optimization.

Goodbye, P-Series! Hello, H-Series?

Intel has simplified its naming scheme a bit with Meteor Lake, removing the "P series," which has only been around for a couple of generations, and wrapping that into the long-existing H-series. Now, there's just H and U.

H series is the larger chip, with a 6+8 die (6 P-cores, 8 E-cores, though there are processors with 4 P-cores), while the U series is designed for 2+8. 

Perhaps the biggest difference is that the H series is running at 28W, which the P series ran at. On Raptor Lake, the H chips ran at 45W. When the Intel Core Ultra 9 185H launches in early 2024, that will be a 45W part. So Intel still has a range of TDPs. It's simpler in naming, but not inherently in power.

"In [the] Meteor Lake generation it will be largely what was our P series, so that thin and light performance — thin and light concept — packing a lot of power into an integrated graphics system," Rogers said.

But that means that people could, in theory, go into a store and think they're getting a different amount of power. How, I asked, will Intel make it clear what you're getting?

"Part of improving communication is simplifying, so we definitely got the feedback and we wanted to just simplify the number of choices," he said. So that's why we made it from a three-letter structure to a two-letter structure which we think is helpful."

Rogers said that the decision will really be made when a customer goes to a store and looks for what they want to buy. That messaging, he said, will come from the company that makes the laptop and what they offer in their lineup.

"And then as a user, you're just able to kind of see, 'OK, what are my needs,' right? Do I want lighter, do I want a little bit more performance? So we have a letter that gives that choice, but really that's a choice that you're going to see in the exact laptops that you're interested in purchasing. So if you know that you're looking for a more powerful [PC], lots of those laptops generally will have H-series."

9W parts and "very different" form factors

Intel didn't have a low-wattage part available in 13th Gen mobile, which led to 12th Gen chips in foldables released this year. In Intel's list of chips coming in Q1 of 2024, there's the Core Ultra 7 164U and Core Ultra 5 134U, both of which are 9W processors. So what new form factors could Intel be trying to enable by going back to low-power parts?

"Without pre-announcing from my friends, colleagues, and partners I think you will see some, notable, very different form factors for Core Ultra, not necessarily in the PC space," Rogers said, pausing.

"It's a PC, but in a different category, let's say," he resumed to finish his thought. I couldn't help but think that every major gaming handheld so far has used AMD Ryzen processors. There's also Lenovo's rollable prototype from Mobile World Congress. At the very least, we could get more modern foldables.

But most Meteor Lake designs will be two-in-ones, clamshells, and other more traditional shapes, he said. "Most of the focus of our partners is actually driving more compute into thinner, lighter envelopes. Thermal technologies have gotten a lot better." Rogers suggested that the increased thermal capacity means you can put a 28W part in a system you used to put a 15W CPU into.

And because AI can be computationally demanding, Rogers predicted that you'll see more computers with the higher TDP choice (the 28W H-series) rather than the lower-power 15W U-series chips, though both will be available.

"At the end of the day, OEMs make the right decision based on their form factor, but we'd like to provide choice," he said. "And I think the biggest thing that I'm most excited about in Core Ultra, for me, is the battery life improvements." All of the data Intel showed at launch, he pointed out, was on the 28W part.

We'll see what Intel and laptop companies come up with over the next year. While a handful of devices from MSI, Lenovo, Dell, HP, Asus, Acer, Samsung, Gigabyte, and more launched this week, CES 2024 is right around the corner. And what would a new year be without a slew of new laptops? We'll see then.