IDK for sure if you need ECC RAM. I don't think so, but with such a computer, maybe it'd be important. If so, then I'd probably also have to throw in a much more expensive motherboard.
This was the cheapest great-quality ECC 4x8GB kit that I could find and it's about twice as expensive as the kit currently in the build:
http://www.amazon.com/gp/offer-listing/B0080K692U/ref=d...
Not a horrible price honestly (slightly better price per GB than I paid for my 16GB of inferior desktop memory last year), but it's nothing in comparison to that of the desktop kit. If ECC is necessary, then that's definitely a sacrifice worth making since my build is currently only a little over half of the $2000 budget that you gave anyway.
SSDs are incredible boot drives. I'm not sure of how much it will help your system because I'm not sure of exactly how you'd go about your task with it such as what distro you'll use and what it's storage characteristics are, but any OS that isn't primarily in the RAM such as TinyCore Linux will probably benefit from having an SSD boot drive. However, SSDs have quite limited numbers of writes before their flash cells wear out and although this isn't a problem for most consumers, it might be for you if you write a lot of data to the SSD. Because of that, if you go for an SSD, you'd probably be better off with either an older model (NAND flash die shrinks that happen every year or two reduce cell endurance dramatically every generation, so old generations can be exponentially more enduring than most newer models) or an eMLC/SLC model.
That means that you'd be paying a lot of money for very little capacity because both options are extremely expensive. For example, even a slower, low capacity eMLC model such as the Intel 710 100GB goes for about $400.
However, if you're not writing a ton of data to the SSD, then you'd be fine with a much cheaper modern MLC model such has a Samsung 840 Pro. The 120GB model of it can be found several times cheaper than the Intel 710 100GB. As a boot drive, I wouldn't expect you to write a whole lot of data, but I don't know what your specific needs for the boot drive would be.
Honestly, I hadn't thought of a backup PSU. This one should be fine, but if you think that you should have a more redundant power supply setup, then I can look into that instead.
Yes, this is still an extremely low power build. The RAM doesn't use much power, SSDs use very little power, and the same is true for all of the components. Even the high-performance CPU and the hard drives aren't very power-hungry at all.