Seeed Studio debuts Raspberry Pi 5 PCIe 3.0 HAT with dual M.2 slots for $45

A PCIe switch is definitely the way to go, but the Pi's SoC really needs to expose more lanes.

The article said:

if you have higher hopes for a proper mini NAS, the yet faster four-slot CM3588 computing module may serve you better.

Very nice! The RK3588 should have PCIe 3.0 x4. I wonder if that board also uses a PCIe switch, but I'd bet it just devotes one lane per SSD. Considering you're limited to just 2.5 Gbps Ethernet, that's more than enough.

P.S. @TheyCallMeContra , it'd be nice if you included a pic of the underside of Seeed Studio's hat (visible on CNX's page), which shows the PCIe switch chip.

abufrejoval said:

The RK3588 itself has tons of I/O, way more and more diverse than the Broadcom SoC inside the Raspberry PI 5 (RP5).

Thanks for the detailed rundown!

abufrejoval said:

for the OP5+ I managed to do with a tiny Noctua fan in a metal case running slower on 3.3V without overheating at peak clocks and I/O loads. It's not silent but unnoticeable, which is good enough for me.

Which case?

Also, what Linux distro do you run on it?

abufrejoval said:

these ARM minis are used as µ-servers in the home-lab.

Once Hardkernel added support for in-band ECC on their ODROID-H4, I went that way for micro-servers and haven't looked back. The H4 models can idle at very low power and have built-in 2.5G Ethernet, which is enough for what I need. Their M.2 slot is full PCIe 3.0 x4, which is actually somewhat uncommon for Alder Lake-N boards. While you can run them fanless, I don't. At idle, the 92mm PWM fan I used is inaudible.

Have you seen the Radxa Orion O6, based on the Cix CD8180 SoC? I might just skip the Orange Pi Plus and go with it, since it has twice as many A7xx cores and they're at least 4 generations newer!
https://radxa.com/products/orion/o6/
They're not yet shipping, so I'm going to wait and see what kind of real world experiences people are having with them. Radxa has an English language user forum, which seems like probably the best place to find out how it works in practice.

I don't know much about Radxa, but they've been around for a while and some of their boards seems reasonably popular. The thin mini-ITX form factor is great for me, because I've got this Silverstone PT13 case that I equipped with a 92 mm fan that should be perfect for it (not the case I'm using for the ODROID-H4):
https://www.silverstonetek.com/en/product/info/computer-chassis/PT13/

abufrejoval said:

Something off Aliexpress or Amazon, just a basic stamped, bent and painted sheet metal case (most seem to be clones) that already came with a fan I found too noisy. The only significant change was to swap to a Noctua and to use 3.3V from the GPIO instead of 5V from the fan header to keep the noise down.

The board doesn't even have a PWM fan header?

abufrejoval said:

Well, actually I also added a heat sink with tall fins to the SoC (had to file off some aluminum to make it fit)

Is your heatsink pure aluminum? If so, you might find that you get better cooling performance by inserting a copper shim between it and the SoC.

abufrejoval said:

the main disadvantage would be dust sucked into the case where that can't happen with the RP5.

I've had the idea to get some washable air filter material and glue strips of Velcro along the edges, in order to make a removable, washable dust filter I can attach over the intake fan of my mini-PC's case. Haven't done it, yet.

abufrejoval said:

even on a 3GHz overclock with a load of power viruses.

What's your favorite power virus? On my N97, the best one seems to be stress-ng --cpu=N --cpu-method=fft although I haven't done a similar search for power viruses on any ARM cores.

Also, on my N97, a mysteriously intensive way to stress the iGPU seem to be: glmark2 -s 1536x1152 -b buffer:columns=200:interleave=false:update-dispersion=0.9:update-fraction=0.5:update-method=subdata --run-forever
In terms of power consumption, it's much worse than any of the other test cases in glmark2 --benchmark .

When I get both of them going at the same time, my N97 mini-PC exceeds 60 W at the wall, which is sort of crazy for a SoC that's supposedly just 12 W. It would thermally throttle way before that, until I put quite some effort into improving its cooling. FWIW, this is a different board - not the ODROID-H4 (which actually uses a N305).

abufrejoval said:

For the OrangePI I had also tested Armbian, which seemed to be almost identical originally, except that the names for tools and files were all changed a bit: I stuck with the OrangePI variant initially, because their documentation was better.
Since then it seems that Armbian has done better on GPU integration, where OrangePi's Debian is missing Mali GPU support, or at least the OpenGL parts: still looking for some time to test that before potentially making the change.

Yeah, I thought one of the bigger differentiators of Armbian was how it's meant to ship with proprietary drivers for the GPU and stuff.

AFAIK, the in-tree support for ARM GPUs (which is all that Debian would have) is still lagging badly behind Intel and Qualcomm. At least, it should be better than Imagination.
🥲

abufrejoval said:

Having a team from Slovenia in charge of the OS these days seems slightly safer, even if the SoC is still very much Chinese.

Yeah, I don't really mind using a Chinese SoC, as long as I can use a non-Chinese distro. For SoCs, there aren't a lot of other options. MediaTek has been dabbling in this space, but I only see the Genio 1200 in higher-end boards. NXP has some ARM-based SoCs, but they're not very competitive with the better Chinese options and seem more oriented towards industrial use cases. Amlogic at least has US headquarters, but they haven't launched anything very competitive since the S922X. Since then, they've seemed to focus more on low-end stuff.

abufrejoval said:

Instead I am still mostly using hardware RAIDs on Windows server for HDD data at rest, all data on SSD either has a HDD or cloud backing store.

I use Linux software RAID. Yeah, it has that pesky write hole, but that's what backups are for. Performance-wise, there's no reason not to use SW RAID, if your devices are mechanical disks or SATA SSDs. Even my old AMD Phenom II could basically max out the media rate of the disks.

abufrejoval said:

Thanks for the link, I'll have a look. But for now, my interest in ARM has declined significantly, I've mostly learned what could be learned from them and for research ARM was closed as a topic with my employer.

I'm intrigued by the prospect of having an ARM mini-PC with SVE that I can tinker with. I suppose I could also use my phone for that, but it's easier if I just have a full Linux install right on the device.

No chance of it replacing my ODROID-H4 as my micro server, though. Not without ECC support, of some kind. I also like that the ODROID-H4 has upgradable/replaceable memory (although speed is limited to DDR5-4800).

abufrejoval said:

Some of the ARM control flow integrity feature stuff of ARM9 simply don't exist in these current SBCs and their SoCs and CHIERI like extensions currently seem to be RISC-V, only.

This post says where you can apply to get a ARM + CHERI (AKA Morello) development board:
https://www.phoronix.com/forums/forum/hardware/processors-memory/1516285-linux-patches-updated-for-experimental-arm-morello-that-combines-arm-cheri-isa?p=1516345#post1516345

abufrejoval said:

stress and stress-ng are both available just the same, as is s-tui to monitor what there is in terms of sensors.
But my favorite is perhaps the simplest to use on nearly all platforms: using Firefox to run a WASM benchmark with a selectable number of threads.Of course, other browsers would work, too, but I prefer Firefox and then it's not about comparing browser engines.

stress-ng has many different CPU methods, and they vary pretty significantly in the amount of power they consume. This is why it bugs me, when I see reviewers like Les Pounder (author of this article) who run it without selecting one of the more intensive options. By default, it just cycles through a bunch of stressors, not all of which are even designed to stress the CPU cores.

I'd encourage you to try playing with the different stress-ng options, if you really want to stress your system to the max. FFT might not be so highly-optimized on ARM. Hardkernel has used --cpu-method=matrixprod, but that's not as good for stressing Gracemont as fft.

abufrejoval said:

iGPU power consumption tends to be pretty boring: they have a rather low fixed upper limit and when you mix both GPU and CPU loads, the GPU typically gets priority. So if you're able to saturate SoC limits with the CPU, adding iGPU won't raise the overall load, just give the GPU it's maximum slice.

What's weird is that Alder Lake-N can seemingly use far more package power than whatever you have PL1/PL2 configured for. At stock settings, it does seem like the iGPU begins throttling the CPU cores, but then it goes on to basically use as much power as it wants. Maybe this is just a weakness of the Linux driver.

abufrejoval said:

For x86 GPU/CPU TDP sharing is easiest to observe via Furmark/Prime95 on Windows, so I don't bother much about GPU tests on Linux. I use glmark2 mostly for some basic functional testing there.

Weirdly, I tried furmark on Linux and it's really not as good at loading the Xe iGPU as that glmark2 commandline I mentioned. From what I recall, it's not CPU-bottlenecked, either. Maybe it's bottlenecking on Alder Lake-N's 64-bit memory interface? You used to be able to query the memory throughput with intel_gpu_top, but I think that broke a while back.

abufrejoval said:

My main interests is stability and being able to live with them in the room: I'm not trying to break speed records.

Yeah, I put a lot of effort into my N97's cooling, simply because I wanted the fan to be inaudible, unless I slammed it with a heavy workload. Originally, I had wanted to make it fanless, but that wasn't going to happen in a form factor like the case I wanted to use.

abufrejoval said:

That's Alder Lake for you. With all the Goldmonts, TDP limits were still pretty much respected.

Eh, I admittedly don't have much experience with it, but in the little time I spent toying around with an Apollo Lake and Gemini Lake ASRock (consumer-oriented; not industrial) mini-ITX boards, they would readily gobble well more power than spec'd.

abufrejoval said:

Of course your power supplies may not be nearly as efficient as you thought and then mainboard components and USB devices can ask for quite significant amounts of power, too, even RAM can want quite a bit from the bottle.

The PSU is a 60 W Seasonic power brick that claims 89% efficiency. They no longer make them, sadly. As for the other stuff, the system idles at around 7.5 W, headless (9W with KDE desktop and 1080p monitor via HDMI). That board is an industrial motherboard (Jetway brand) with a bunch of serial ports, a SATA controller, 2x 2.5G Ethernet, but not much else. The SSD is SK hynix P31 Gold (500 GB) and the M.2 slot is PCIe 3.0 x2. The RAM is a 32 GB DDR5-5600 (SK hynix die), but running at only 4800.

Hardkernel's ODROID-H4 has better idle power consumption, but I haven't tested it because I'm still in the setup & migration phase over from the Jetway board. Here's what they claim:
Desktop GUIUbuntu Desktop BootingGnome Desktop GUI IdleCPU stress4K YouTube play on Chrome BrowserWebGL aquarium demo on Chrome BrowserWebGL + CPU StressPower OffSleep (Suspend to RAM)
Headless ServerTestConfigurationUbuntu Desktop GUI
(Power-Save Governor)HDMI connectedHDMI disconnectedUnlimited Performance mode On (PL4=0)Off (PL4=30000, Default)PCIe ASPM option in BIOS settings All Disabled (Defaut)All AutoEthernet connection YesNo
Source: https://www.hardkernel.com/shop/odroid-h4/

abufrejoval said:

No, the main advantage of hardware RAID is management, OS neutrality and that I already owned the hardware.

I don't use an OS-neutral filesystem and I like being able to take my disks and read them on any PC running a modern linux build.

abufrejoval said:

But that hardware is now more than 10 years old, also eats power and the spares aren't as drop-in compatible as they should be and need their own maintenance and testing to ensure their readiness.

I used to like the hardware RAID on Dell servers, until their backplanes started failing (admittedly, these are older models, long past expiration of their warranty). I had some repurposed servers at work that I was using to host some shared filesystems, and it wasn't until the second one failed that I figured out it wasn't actually the disks that had gone bad.

abufrejoval said:

So I am testing all types of software RAIDs currently, including storage spaces and ReFS on Windows.

When it comes to storage, I'm a believer in KISS (Keep It Simple, Silly). The more widely-used and standard configuration, the less likelihood of encountering problems.

abufrejoval said:

becasuse I had the audacity to boot a Windows 2025 server to Go on it.

No the Storage Spaces pool was not upgraded, but unless you prohibit that via a registry entry, ReFS file systems will up transparently upgraded to any newer OS... Microsoft just cannot imagine that a Windows 11 or Server 2025 might just be temporary...

I never trust Microsoft not to mess up any other filesystems or operating systems, when I do an install or upgrade. I always disconnect all other storage devices, when I do those things.

In my experience, Linux installers are always good about at least telling you what they're going to do, before they do it.

abufrejoval said:

I really want to run a mix of Proxmox and Univention at the base, with some hot storage on Ceph, some hot storage via pass-through on NVMe and bigger data-at-rest via ZFS with replication and available via SAMBA and NFS4.

I'm just using NFS + Samba and use rsync for backups. I keep it simple.