Custom built, sort of. Most supercomputers do use off the shelf components, just not consumer level off the shelf components. What is almost always custom is the programming/OS that runs it.
Off the shelf PDU, racks, blade servers, network infrastructure (these days they are using 10Gbps or higher).
CPUs/Boards/Blades are either off the shelf or custom. Lot of servers just have piles and piles of Intel Xeon or AMD Operton blades.
I know China has recently been branching off to make their own ARM based super computers. And you still have suppliers like Cray, IBM, etc that provide commercially available 'supercomputer' components or complete systems.
For simulations, a lot of companies are turning to GPGPU farms with hundreds of the highest end Nvidia and AMD 'Math' cards.
Really it is up to the intended use to set how they go about building it, and availability of hardware. I know the US government has basically taken almost all of Nvidia's production of their new NVLink equipped hardware.