The developer preview of DirectML DirectX12 extension and Windows' machine learning API has added support for Intel Core Ultra CPUs and their NPU AI hardware as of the release of DirectML 1.13.1 and ONNX Runtime 1.17. DirectML's general purpose is to provide a low-level API and abstraction layer for machine learning accelerators (Direct "Machine Learning"), whether discrete GPUs or integrated NPUs. Thus, adding support for Intel's Core Ultra NPUs was pretty much just a matter of time.
According to the original Windows developer blog post and the related Intel blog post it links to, this release was developed with Samsung's assistance and open source models. According to Hwang-Yoon Shim, VP and Head of New Computing H/W R&D Group at Samsung, "Windows DirectML is one of the most efficient ways for Samsung's developers to make those (efficient machine learning) experiences for Windows."
While this initial release of NPU support for Intel Core Ultra CPUs is a landmark for DirectML, it does come with many compromises as a developer-first release. Limitations listed on the original page dictate that only some machine learning models are supported by DirectML and that only Intel Core Ultra NPUs will be supported for this particular release. According to Microsoft, machine learning models excluded were excluded due to stability or accuracy issues, including issues like not running at all.
Improved support for the Intel Core Ultra NPUs on Windows should help push the image of so-called "AI PCs" forward, which pretty much describes regular PCs with onboard AI acceleration hardware of some kind. While GPUs have, of course, been doing AI acceleration since the initial release of Nvidia RTX in 2018, the latest CPUs have only just started, so it'll take a little time before all the involved software catches up.
As Intel Core Ultra CPUs continue to roll out with the release of Intel's Meteor Lake architecture, support for Core Ultra's onboard AI hardware should continue to improve. Meteor Lake has more to offer than just improved AI performance, though— general power efficiency and especially iGPU performance have also improved on the new architecture compared to Intel's last few generations.
