AMD Asks: Do You Need Ryzen AI Support in Linux?
Currently supported only in Windows.
Laptops based on AMD's latest Ryzen 7040-series processors come equipped with Ryzen AI engine, an accelerator for machine learning applications. However, this feature is currently exclusive to Windows, leaving Linux users hoping for broader compatibility. Community interest has prompted AMD to reconsider, an reopening a GitHub ticket for feedback and expressing willingness to support the technology if there is adequate demand, reports Phoronix.
AMD has designed the Ryzen XDNA AI engine for less demanding AI inference tasks like audio, photo, and video processing. Its goal is to provide quicker response times compared to online services, and it is also more energy-efficient compared to solutions based on CPUs or GPUs. The engine has the capacity to manage up to four simultaneous AI streams, and it can process INT8 and bfloat16 instructions.
According to AMD, the performance of this engine surpasses that of the neural engine in Apple's M2 processor. AMD's Xilinx-based AI engine is compatible with popular frameworks such as TensorFlow, PyTorch, and ONNX, but the problem is that the company's Ryzen AI Software Platform version 0.8 only supports Windows. By contrast, Intel has already integrated open-source AI processor support in Linux.
This limitation has sparked a discussion within the tech community, particularly among Linux users who desire the same advanced capabilities on their systems. AMD has been receptive to these discussions, enabling a platform on GitHub for users to express their interest and thoughts on Linux compatibility.
Taking user feedback seriously, AMD has demonstrated flexibility and openness to expanding Ryzen AI’s compatibility based on customer demand to Linux. A GitHub ticket that allows users to voice their need for Linux support has been reopened by an AMD staff member, signifying the company's willingness to listen and potentially act based on the community's needs and interests.
The only question is when will AMD be able to bring proper support for Ryzen AI to Linux. The company yet has to release final version 1.0 of its Ryzen AI Software Platform for Windows, which is dominant in the realm of PCs. As a result, the it's almost certain that the majority of the company's effort will be dedicated to Windows, not Linux, in the coming months.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.
-
bit_user Since a lot of AI development happens under Linux (dare I say even the bulk of it?), this would be a good move for developer engagement and mindshare, if nothing else.Reply
AMD has designed the Ryzen XDNA AI engine for less demanding AI inference tasks like audio, photo, and video processing. Its goal is to provide quicker response times compared to online services, and it is also more energy-efficient compared to solutions based on CPUs or GPUs. The engine has the capacity to manage up to four simultaneous AI streams, and it can process INT8 and bfloat16 instructions.
Good summary. The key point is that it's not faster than the GPU. AMD actually told us how fast it is:
Ryzen AI can reportedly do up to 5 TFLOPS (BF16)
780M iGPU reportedly good for 8.6 TFLOPS (FP16).
So, this AI accelerator isn't going to be a game changer for generative AI. It's really about extending battery life, when using AI for things like video conferencing or video playback. -
bit_user Reply
Feel free to mention that on the github issue (if you haven't already):unis_torvalds said:Agreed. Linux support should be a no brainer.
https://github.com/amd/RyzenAI-SW/issues/2#issuecomment-1773335415
The more votes, the better! -
tracker1 Linux is definitely necessary. Almost nobody is targeting server loads under windows for this kind of work. It's all Linux there.Reply
I'd go a half step farther and say that dev ex should have parity with WSL as well as Linux hosts. Since a lot of projects resetting Linux are done in Windows with WSL. -
bit_user Reply
Well, I don't know how much power it uses, but Chips & Cheese pieced together some details from their Hot Chips presentation + details Xilinx has published about them. Scroll about half way down this page:abufrejoval said:It's very hard to say, when there seem to be no details on
functional scope: what can that NPU do?
comparative computing power: how does it compare to say a Hexagon DSP/NPU?
power/performance/usability: how much power will it require at which type of ML workload, can it be regulated to meet latency expectations?
integration with the rest of the SoC: how much CPU collaboration/wake-up will be required e.g to do voice recognition and command processsing?
https://chipsandcheese.com/2023/09/16/hot-chips-2023-amds-phoenix-soc/
Support as in what? Their prepackaged drivers? You'll still have the open source driver supporting them. It's not as if ROCm ever supported them, so no loss there.abufrejoval said:I can tell AMD what I don't want to go and that is iGPU support for my Cezanne APUs, which are getting awfully close to being jettisoned as near all GCN GPUs loose Linux driver support these days: two years of support is quite ridiculous for an APUs iGPU which you can't swap out!
I might agree with you, if I better understood what you're talking about. -
bit_user Reply
Huh. Why didn't you just link the phoronix post?abufrejoval said:In response to this post.
https://www.phoronix.com/news/AMDVLK-2023.Q4.1-ReleasedAs that article says:
" You can either stick to AMDVLK 2023.Q3 and older or the better option is to simply use the community-maintained Mesa RADV driver."So, I think it's a bit of a non-issue. I don't quite know why AMD even keeps maintaining AMDVLK, but I guess they just like to have something they control, so they can theoretically be sure it provides the best support for new hardware at launch & supports all the latest & greatest features of their hardware, without having to negotiate or compromise anything with the Mesa developers.
The best thing about your link is this pic, which is just like... weird. As if they gave it to her to pose with, yet she doesn't really know quite what it is.
-
bit_user Reply
AMDVLK is a userspace component, thus probably not involved in power management. Just use RADV and you'll be fine.abufrejoval said:And in the case of Linux GPU drivers, it's mostly the power management features for SoCs that are important to me. And that's one area where open source drivers have suffered from lack of technical vendor support. Again, I'm not trying to game on notebooks, but trying to remain productive during travel or where I can't keep people staring at their mobile from stepping on my power line. -
hwertz First off, I will comment, although Intel now says the.. 14th gen? Intel CPUs are the first with an AI accelerator, I have an accelerator of some sort in my 11th gen CPU. It's VERY limited, I think in Windows it's used to clean up mic input. It's got full Linux support, but due to the limitations on input format (1D data only, and a small size at that) and data formats (maybe int8 only?) I found when I was dicking around with tensorflow and such that it was not actually useful for anything for my use.Reply
So...
1) I find it EXTRAORDINARILY odd that AMD would even mull having Windows support and no Linux support, as others have said SO MUCH AI stuff is done in Linux. I mean, Nvidia supports both but it's clear the premier environment is Linux.
2) That said, with int8 and float16 support, it may be rather limited for people thinking they can just fire up tensorflow and run whatever on it. It sounds like it's less limited than the thing I have in my 11th gen Intel system, but still I'm not sure how many tensorflow tools will work with only int8 and float16 data types. -
bit_user Reply
Yes, I've mentioned it in other threads.hwertz said:First off, I will comment, although Intel now says the.. 14th gen? Intel CPUs are the first with an AI accelerator, I have an accelerator of some sort in my 11th gen CPU. It's VERY limited, I think in Windows it's used to clean up mic input.
https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_supported_plugins_GNA.html
As you mentioned, it's not very interesting, due to the small amount of compute power it provides. The main benefit it provides is improved energy efficiency, by offloading things like audio processing or presence detection from the CPU or GPU.
AMD is a Windows-oriented company. It's becoming less so, but their stuff has generally had better support under Windows than Linux.hwertz said:1) I find it EXTRAORDINARILY odd that AMD would even mull having Windows support and no Linux support,
Those data formats are fine for most inference acceleration, which is the point of these integrated NPUs. If you want to do training, you really need to use a dGPU or bigger.hwertz said:2) That said, with int8 and float16 support, it may be rather limited for people thinking they can just fire up tensorflow and run whatever on it.