Why you can trust Tom's Hardware Our expert reviewers spend hours testing and comparing products and services so you can choose the best for you. Find out more about how we test.
I was hoping to expand both the set of inference engines and AI models tested for this review, but I quickly ran into issues. The Ryzen AI Halo ships with a version of vLLM pre-installed, but it wasn't compatible with Qwen 3.6-35B-A3B when I tried to launch it, so I fell back to the old reliable llama.cpp.
We tested three models using llama.cpp: Qwen 3.6-35B-A3B, a relatively lightweight mixture-of-experts model that's been extremely popular of late, Google's Gemma 4 12B, another recent and relatively lightweight but dense model that activates all of its parameters per token, and gpt-oss-120B, a larger mixture-of-experts model that's been available for quite some time now. We used Unsloth's GGUF versions of these models in their Q4_K_M quantizations.
This time around, we're using llama-benchy as our benchmark harness. llama-benchy lets us get llama-bench-like performance results out of any model runner that can present us with an OpenAI-compatible endpoint, not just llama.cpp. That’s quite handy for comparing performance across inference engines.
First up, we'll look at latency and throughput performance with Qwen 3.6-35B-A3B:
And with the dense Gemma 4 12B:
And finally, with gpt-oss-120B:
At least with llama.cpp, the AI Halo's relative performance in single-user LLM serving versus GB10 isn't any different than what we saw several months ago when we took the Corsair AI Workstation 300 through its paces. While its tokens-per-second throughput is fine, albeit still slower than the Dell Pro Max GB10, its time-to-first-token latency falls far behind GB10 as context length grows.
In a real-world scenario where llama.cpp can use (and is configured to use) its prompt caching features, these differences might not be as pronounced, but it's still important to note this worst-case behavior, especially for long-running coding workflows where context lengths can quickly grow.
Overall, the Ryzen AI Halo doesn't work any magic for Strix Halo inference performance compared to other implementations of this platform we've tested. While its tokens-per-second throughput is acceptable, its time-to-first-token latency can quickly rise to non-interactive levels with long contexts.
At the extremes of our testing, waiting two to four minutes for a model to start responding might be disruptive to an interactive workload like a coding assistant, even if the rate at which tokens flow is tolerable once they do start rolling.
I also ran the same check-in on ComfyUI image generation performance using the same basic Flux.2 Klein test we ran back in February, as well, and the same gulf that we saw in time-to-completion for ComfyUI work remains now. The GB10 GPU's larger shader complement chews through image generation far quicker than the Radeon 8086S.
Finally, we checked in on CPU performance with Geekbench 6. The AI Halo’s 16-core, 32-thread Zen 5 CPU holds an edge on the GB10’s 20-core Arm CPU complex in both single-threaded and multi-threaded performance, so tasks like code compilation could potentially be faster on the AI Halo. But that victory shouldn’t cause us to lose sight of the GB10’s all-around better AI performance.
Thermal performance and noise levels
We didn't want to tear down our Ryzen AI Halo to reveal its cooling system, but as we discussed at the beginning of this review, the system has vents on its front, top, and sides to allow for plenty of airflow, and you can see a decently sized copper heatsink through its rear vents.
Logging system temperatures on Linux is more difficult than it is on Windows, but we didn't see CPU or GPU temperatures higher than the mid-50 °C range when running the AI Halo through our typical workloads. All that suggests that this box is more than up to the task of cooling the chip inside.
As for noise, the Ryzen AI Halo runs its twin blower fans audibly even at idle, so it's always adding some amount of noise to a room. And under a ComfyUI generative workload, it gets significantly louder than the Dell Pro Max GB10 box we're using to represent that platform.
The noise signature of those fans is also a bit less refined than those in the GB10 boxes I’ve used. They have a notable high-pitched whine that's difficult to acclimate to when the system is placed on a desktop, whereas the GB10 systems I've used all just sound like moving air and are easy to ignore.
- MORE: Best Graphics Cards
- MORE: GPU Benchmarks and Hierarchy
- MORE: All Graphics Content
