Requesting feedback on NAS build

Hi all!

I want to build myself a little home NAS, and although I have done a lot of reading over the past few days, and I'm starting to have a clearer idea of what I do and do not need, I'd still like to have a bit of feedback from other people on the topic...

What are my goals?

• 
  ■ Ability to hold several (4, ideally 6) 3.5" hard drives
  ■ Ability to swap out a drive without too much pain
  ■ Optional: ability to extend this with an extra box, e.g. through eSata connection
  ■ No hardware RAID
  ■ Some degree of resiliency to hardware failures
  ■ Usage will be relatively scattered, so low idle power consumption would be great, and ability to go to sleep and wake-up quickly (< 2 seconds) through Wake-on-lan would be ideal
  ■ Can be administered remotely (no need to plug in keyboard, monitor, etc beyond the initial setup)
  ■ Ability to run some custom services (e.g. OwnCloud), and perhaps script it a bit
  ■ Not too big, so long as it can fit the number of drives above (no need for graphics card, and no need for optical drive beyond initial setup, should free up some room inside)
  ■ I tend to add hard drives slowly, so I'd rather not go for a setup where I need to add, say, 4 drives at a time to expand storage (example of a ZFS pool with RAIDZ2). Adding drives one by one, or two by two, would be better

I don't want to go with a pre-packaged NAS, because 1) it's expensive, 2) it's less fun, and 3) it's not as customizable (hardware and software).

I don't want to go with cloud storage (Backblaze, Crashplan, Dropbox, ...) because 1) it's expensive in the long run, 2) it's less fun, 3) my internet connection is not all that fast, and 4) I want to retain control of my data. I will still use some form of cloud storage (probably Google Drive) as a repository for backing up some or all of that data though.


Software stack

The remote administration + scripting + hardware failure resiliency requirements point to a Linux/BSD-based distribution. On top of that, it would be free.

I've narrowed it down to the following candidates:

• 
  ■ FreeNAS with ZFS
  ■ NAS4Free with ZFS
  ■ (headless) Ubuntu with ZFS (I believe it now ships with stable, production-grade ZFS support by default?)
  ■ Ubuntu with MADM

I think Ubuntu is a better choice for me, because 1) I have some experience with Linux administration (Debian, Ubuntu), but not so much with BSD. And 2), it should be easier (although a bit more work) to run other services - I'm not limited to whichever FreeNAS / NAS4Free plugins happen to be available.

I could always slap Webmin (or an alternative: Ajenti, ...) on top of it to make remote management more pleasant. Feedback appreciated on that topic!

As to the storage layer, my options (under Ubuntu) are mdadm RAID 5/6/10 + some journaling FS (JFS? XFS? ext3? ext4?), or ZFS mirror/RAIDZ/RAIDZ2. My understanding is that RAID 5 and RAIDZ are to be avoided, due to the long resilvering times, and the likelihood of cascading failures resulting in total loss of data. ZFS has a number of advantages over madam RAID + any FS, including checksumming and error detection.

So my top choices seem to be RAIDZ2 or "RAID 10" (pool of mirrored vdevs).

Pros of RAIDZ2:

• 
  ■ Better space efficiency with 6 disks (66%, vs 50% for RAID 10)
  ■ Slightly more resilient with 4 disks (any 2 disks can fail)
  ■ Only 4-6 disks means that resilvering shouldn't take that long
  Pros of RAID 10:
  ■ More resilient with 6 disks
  ■ Only need to add 2 disks to expand the pool. Since ZFS cannot add disks to existing vdevs, growing pool with RAIDZ2 is going to be much more expensive (at least 4 disks, whether I'm adding a new vdev, or upgrading all the disks of an existing vdev).

I'm thinking that ZFS "RAID 10" is probably the best option for me; the ease of upgrade probably outweighs the small storage size penalty. Thoughts? Any reason I'm not seeing to choose mdadm RAID 6 or RAID 10 instead?

A word on memory: I know that there is a lot of debate about how bad it really is to run ZFS without ECC memory. From what I've seen, while ECC is certainly a good idea, it is not an absolute requirement of ZFS (at least, not moreso than other FSs). If I can get my hand on a ECC RAM, and a CPU + motherboard that support it, all of it at a reasonable cost (i.e. maybe another 10-20% budget, but not doubling the budget), I'll certainly consider it.


Now that the software part is defined, I have two options for hardware:
1) Repurpose old parts
2) Build it from scratch (apart from hard drives, which I already have)


Repurposing

I have two computers laying about that I am not using, with full ATX cases. Here's the list of components I have:

• 
  ■ 2x Antec 300 (http://www.newegg.com/Product/Product.aspx?Item=N82E16811129042)
  ■ ASUS M4A79XTD EVO AM3 AMD 790X (http://www.newegg.com/Product/Product.aspx?Item=N82E16813131402)
  ■ AMD Sempron 145 Sargas Single-Core 2.8 GHz Socket AM3 45W (http://www.newegg.com/Product/Product.aspx?Item=N82E16819103888)
  ■ 4x Seagate Barracuda 7200.12 ST31000528AS 1TB SATA 3Gbps (http://www.newegg.com/Product/Product.aspx?Item=N82E16822148433)
  ■ Patriot 2GB 240-Pin DDR3 1133 (http://www.newegg.com/Product/Product.aspx?Item=N82E16820220453)
  ■ Antec EarthWatts EA650 650W 80 Plus (http://www.newegg.com/Product/Product.aspx?Item=N82E16817371015)
  ■ Intel Core 2 Quad Q9450 2.66 GHz 95W (http://www.newegg.com/Product/Product.aspx?Item=N82E16819115042)
  ■ ARCTIC Freezer 7 Pro Rev. 2 CPU Cooler (http://www.newegg.com/Product/Product.aspx?Item=N82E16835186134)
  ■ ASUS RAMPAGE FORMULA LGA 775 Intel X48 ATX (http://www.newegg.com/Product/Product.aspx?Item=N82E16813131284)
  ■ 2x G.SKILL 2GB 240-Pin DDR2 1066 (http://www.newegg.com/Product/Product.aspx?Item=N82E16820231166)
  ■ Antec NeoPower 650 650W (http://www.newegg.com/Product/Product.aspx?Item=N82E16817371011)

On the upside, that means I could build something right now at zero cost.
On the downside:

• 
  ■ Big (full ATX case, full ATX motherboard)
  ■ Less power efficient (CPU, PSU)
  ■ No ECC memory
  ■ Limited bandwidth on SATA ports
  ■ I could repurpose at least one of these computers anyway to replace a 10-year-old desktop that is way too slow.

I'm thinking I might as well build something new.


Build it from scratch

Based on my requirements, here's my thinking:

Enclosure
There should be enclosures that can fit 6 disks, but are reasonably small (mini-ITX? mini-DTX? micro-ATX? mini-ATX?).

I'd also probably need something to get easy access to the drives - I think I remember reading about "adapters" that e.g. transform 3 5.25" into 5 3.5" disk racks?

CPU
At first I was going for a low TDP core i3 (e.g. 35W), or a Pentium, or an Atom (Avoton?), to help reduce the power consumption. But thinking about it, the TDP matters for the kind of heat dissipation one needs to plan for, rather than normal power consumption. Even beefy processors might idle very low, and if anything, the idle power consumption might be dominated more by the PSU wattage and efficiency, and the motherboard idle consumption, than the CPU. The real advantage of Avoton (and perhaps 35W Celeron, Pentium and Core i3?) is that there is less room taken up by the heatsink/fan due to the lower TDP, so easier to fit in a smaller case. Thoughts?

In terms of choice, I'm thinking either an Avoton (if I find a good board), or a Core i3-6100T / 6300T (35W). If I'm willing to go up to 65W, then the Pentium G family (3900, ...), some Core i3 (6100, ...) and i5 (6400, ...) become possible. The only thing about the Avoton is that I'm not sure the CPU and mobo combo necessarily supports ECC memory - this is supposed to be cheap stuff, after all.

Motherboard
I'd go for something with ideally 6x+ 6.0 Gbps SATA ports, Gigabit ethernet, and 1-2 eSATA ports as well if possible. It'd have to have (basic) on-board graphics (I'll really only need it for the initial setup).

For instance, this seemed like a decent Avoton candidate: http://www.newegg.com/Product/Product.aspx?Item=N82E16813157419... but many reviews where the board died within 2 years! Also, not enough SATA3 ports.
For something that supports Celeron, Pentium, Core i3 (and Xeon!), this looked good: http://www.newegg.com/Product/Product.aspx?Item=N82E16813182995.
There are also a plethora of non-server motherboards (e.g. http://www.newegg.com/Product/Product.aspx?Item=9SIA1N83UD9854, http://www.newegg.com/Product/Product.aspx?Item=N82E16813128871), but they do not seem to support ECC memory, and they tend to come with a bunch of features (HDMI ports, overclocking, audio chipset, ...) that do not make much sense for my build...

There are also a number of LGA 1150 motherboards (Broadwell / Haswell), not sure it's worth looking at? Also, does anyone have experience with SuperMicro?

RAM
I'm thinking 8 or 16 GB of RAM (ECC if possible) would serve well. From what I've heard, ZFS is somewhat memory hungry. It seems that at least the Skylake procs all support ECC, so it's down to the motherboard? By the way, do motherboards that accepts ECC require ECC? Or will they work fine if you feed them non-ECC memory?

OS Storage (excluding RAID array)
I'll have a reasonably-sized SSD (preferably something that can be onbard via PCIe, to save space and SATA ports) to run the OS. I'm thinking this is a better choice than a USB thumbdrive (faster, better suited to this kind of workload).

Note that I am not including SSDs for L2ARC or the ZIL (it will reside in the pool). I will not be turning on deduplication.

PSU
I'm thinking a 250W - 300W PSU should do nicely, and hopefully even at idle the PSU should have decent power efficiency. 80 Plus Gold seems to be my best bet here, as 80 Plus Platinum or Titanium do not seem to be available at these power ratings. Not clear how important it is to strive for maximum power efficiency, given the planned power draw at idle? (although the Platinum spec includes a requirement at 10% power, which is nice) Also, probably worth keeping in mind that not all PSUs are created equal when it comes to providing power when the system is sleeping... but not sure what's the best way to find PSUs satisfying that criterion.

I know there are also microPSUs, but not sure how much of a win it is in this situation - my understanding is that they are great for low-power systems where there is no space physically to put a PSU in the enclosure.

I was thinking for instance this: http://www.newegg.com/Product/Product.aspx?Item=N82E16817151117.



If you've read until here, thanks Any thoughts / feedback on the above?