afifim

Distinguished
Aug 19, 2009
4
0
18,510
Background

I've recently been needing to expand my storage on my local machine, and decided to invest in a RAID solution for home use.
I wanted a number of things though out of this,
■I wanted something that will survive hard disk failures. Touch wood, my 1TB hard disk that I've filled has not failed. But losing over 1 TB of data seems insane.
■I really wanted something with a lot of space. So going from 1 TB to 2, just seemed like the need to upgrade in less than a couple of years.
■DLNA looks like an amazing suite of technologies. The whole point of NAS is streaming media, with DLNA support I can play it back over my PS3 or XBOX 360 that are already hooked up to my T.V.

I looked around, and initially went with the Buffalo Linkstation Quad. RAID5, DLNA, and 3 TBs of usable disk space.
For approximately 4 weeks it worked a treat. DLNA to PS3 worked a treat, although transfer rates were a bit poor if you ask me 10 MB/s (I was on a hundred mbit connection at the time).

After those 4 blissful weeks, streaming to my PS3 just stopped working. I mean everything worked fine, just it did like a frame every 20 seconds or so in standard definition videos, let alone HD...

Two weeks of attempted troubleshooting, and support cases with Buffalo, the Quad has been safely returned to Amazon (kudos to them for accepting a return after 6 weeks), and I am now in the market for a NAs solution again.

So with all of these lessons learned, these are going to be my new requirements

Requirements

■RAID 5 still seems like a must. So minimum 3 drives
■Minimum 3 TBs usable space
■I liked the Buffalo's small form factor. Mini-ITX or smaller is the way forward
■I liked the Buffalo's quiet demeanor, the more silent the better.
■One of the troubleshooting steps was the purchase of a gig-e switch. 10-20 MB/s on a gig-e connection is just taking the piss, specially when I know wire speed is 80 MB/s +, and 3-4 disks can easily sustain that.
■Can not be proprietary. if I can't login and run ethereal + top/taskmgr to figure out wtf is draining the NAS device's performance from DLNA, I ain't using it.
■The days of working DLNA were like a sweet dream, I'd really like them back.

So after a huge amount of research here is what I've come down to

1. For small form factor, and quiet, I want an Atom motherboard
2. For decent speed, I want either a
a. Decent RAID controller that can handle 4 drives, OR
b. 3 or 5 drives so I can setup the block size / stripe size just right, align up the stars, and get my gig-e wire speed.
3. For non proprietary, and DLNA, I want Linux support and PS3 Media server
4. For 3 TBs of disk space I can,
a. Use 3 1.5 TB disks
b. 4 or 5 1 TB disks (I have two already so this would be cheaper for me)
c. If form factor is an issue, do something silly like 2 x 2.5 inch drives in every 3.5 inch drive bay. E.g. 6/8 2.5 inch drives will meet the requirements

Hardware

Case will be Chenbro ES34069. Meets requirement 1, and the largest mini itx case I could find in terms of number of drive bays for requirements 2 and 4
Mobo, I'm really torn on this,
■ Jetway JNC92-330 + Jetway 4x SATA Daughterboard, meets requirements + 5 3.5 inch disks + CF for OS disk. Meets 2b, and 4, assumes I can do RAID 5 across controllers in software, assumes tha Atom RAID 5 in software is decent, and assumes I can somehow magically fit 5 3.5 inch drives in the 4 bay case...
■ MSI IM-945GC-A + 4 3.5 inch disks + CF for OS disk, Meets 2a, and 4. Assumes that software RAID 5 on an ICH7 (note the absence of R in the name) will perform fine. Intel storage controllers are supposed to perform ok even when the stars don't line up.
■Some ION mobo (POV for e.g.), + 3 x 1.5 TB disks + CF for OS disk. Meets 2b, and 4. Assumes that there is an ION board with 3 SATA ports, and an IDE for CF. Good news is there are no physical constraints by the case, but I am back to stars aligning for decent RAID 5 performance.

Note that I had to eliminate decent hardware RAID 5 because of the case size. So the beautiful Adaptec SATA RAID 5 controllers are a no no unless someone can conjure up a better case than the Chenbro.

Anyhow long post over, I'm interested in hearing all ideas about the above, experiences and thoughts. If you don't like my ideas you know what I'm after so please post away.

PS. If you don't know what I mean by stars aligning for decent RAID 5 performance, read this exceptional and amazing post, http://forums.storagereview.net/index.php?showtopic=25786

[Edit]

Updated for drunkness
 
I think your expectations are pretty high and it will be difficult to meet every one of your requirements. I think you should re-evaluate your stand on using a mini-ITX case and a software RAID solution.

By all means use a mini-ITX mobo but consider upsizing to a mid-tower ATX case with a dedicated hardware RAID controller. It seems to me for that if you are going to use this NAS as a long term, upgradeable, and expandable solution, why short change yourself with space constraints. Sure, 3TB is alot of space now, but how about 3 years from now?

I also noticed no mention of what NAS OS you are considering. Given that you are looking to use a CF card as the OS drive, your options are (realistically) limited to Openfiler and FreeNAS; of which I highly recommend FreeNAS.
 

huron

Distinguished
Jun 4, 2007
2,420
0
19,860
+1 Chunkymonster. I think that if you want such a small form factor you'll have a tough time fitting all of that inside an itx case. Why not get a small micro-atx case and fit everything in there?

Another +1 for the NAS OS choices. FreeNAS is an excellent choice.
 

afifim

Distinguished
Aug 19, 2009
4
0
18,510
These are two possible Micro ATX cases,
http://www.newegg.com/Product/Product.aspx?Item=N82E16811156200
http://www.newegg.com/Product/Product.aspx?Item=N82E16811146044

Not sure why you think Mini ITX is out of the question?

I can definitley fit,
3/4 X 3.5" disks
1 X CF + IDE interface

In the Chenbro.

Or is the suggestion that on board RAID 5 performance on any Mini-ITX is really just out the question? Even when the block size / stripe size #of disks is exactly right for synchronous writes to be recognised? I don't completely buy that,

e.g. http://kerneltrap.org/mailarchive/linux-kernel/2007/1/11/44612
e.g. http://freebsdwiki.net/index.php/RAID,_performance_tests
e.g. http://episteme.arstechnica.com/eve/forums/a/tpc/f/96509133/m/641000970931

Tweaked software RAID 5 config should be fine, even on an Atom? Atom's performance sucks for if statements because it is an inorder processor, but for parity calculation that shouldn't matter.

Freenas, I was very tempted because of ZFS, I like the idea of snapshots.

With an IDE or even SATA interface for the CF, Linux won't be able to tell the difference, so that will work fine (or so I say)

I preferred Linux for familiarity, apt-get, and PS3 Media Server (it can transcode), and if not, it streams to both the XBOX360 and PS3, so happy to re-encode all my media to make it work for both if transcoding on an Atom 330 is going to be crap. One of the reasons I've considered an ION mobo is for the potential of doing GPU video transcoding (maybe not easily with current open source software, but certainly down the line).

For OS, I will most likely try both. FreeNAS, test RAID5 performance + DLNA ability. Fallback to Linux if necessary.

Not fussed about expandability. It took me 18-24 months to fill a 1 TB disk, it will take at least 3 years to fill the 2 TB of usable disk space. I tend to do hardware refreshes in 3/4 year increments, and by the time I revisit this, I'm sure disk drives would have doubled in capacity, or new ideas came about for NAS, and another $1,000 doesn't seem too dumb of an idea. (Or just a bigger case, + 2 more 1.5 TB disks for $200 - $300)

So here is an attempted summary,

Case Chenbro ES34069 http://www.mini-itx.com/store/?c=42
Mobo POV ION-MB330 http://www.pointofview-online.com/showroom.php?shop_mode=product_detail&product_id=97
OS Disk Startech SATA-CF adapter, http://www.startech.com/item/SAT2CF-SATA-to-Compact-Flash-SSD-Adapter.aspx + Some bootable CF
Disks 3 x SAMSUNG 1.5TB 32MB ECOGREEN F2

Only real caveat to all of the above is that once tweaked, software RAID 5 performance + Atom 330 will be just fine. Am I wrong on this one?
 
Not totally out of the question, but typically mini-ITX cases are space constrained and require low profile components, i.e.; heatsinks, low profile pci/pcie cards, etc. Heat is the enemy of components and proper case cooling is a genuine concern. That Chenbro case, airflow aside, seems to have the (bare) minimum space your build would require. The removeable/hot swap drive cages are a nice feature.

Tweaking the RAID array depends on the implemenation. If software, the tweaking options may limited whereas a hardware controller typically offers more functionality, depending on the brand of card.

Again, I highly recommend FreeNAS, it is the bomb-diggity!

FreeNAS offers the same functionality, is plug and play with Xbox and PS3, and has built in CFS/SAMBA support along with FTP funtionality all built into the kernel.

True, Linux (Openfiler and FreeNAS) will not care if it's on a CF card.

Even on a 3/4 year refresh schedule, if you use Openfiler or FreeNAS, you will not need to upgrade the base hardware, the most you would need to upgade are the drives for increased size. Another plug for a hardware controller and a micro-ATX is online capacity expansion; the idea being that if in 3/4 years you need to increase the size of your array, you can add another drive and use the controller to expand the size of the array. This approach will save the steps of having to off-load/back-up/image your data, swapping out the drives, recreate the array, and then copying/re-imaging all your data back to the new array.

Nice hardware selection!

I can not speak to the RAID5 functionality/performance of Linux/Openfiler, but given that I am running FreeNAS for my own NAS, I do not recommend the software RAID5 implementation built into FreeNAS.

I don't offer these opinions to knock your idea but to provide another POV as well as other possible approaches to build your NAS. Hopefully this sharing of ideas helps you consider other options and view your solution from a different angle. Experience has taught me that hardware RAID solutions ulitmately perform better and you can never have enough space inside your case.

Regardless, your approach will work and end up being one heck of a NAS. Good luck!
 

afifim

Distinguished
Aug 19, 2009
4
0
18,510


:), the whole reason for the post is for someone to point out why the theory I have is wrong. So please keep at it.

So in summary



Cooling may be a problem in the mini ITX case



Software RAID 5 may not quite work. Definitely my concern as well.



Agreed, giving back that Buffalo was tough enough, cramming my 1.2 TBs of data over 1 TB disk, and 1 500 GB disk that are in use was not fun. Specially with 10-15 MB/s transfer speed. Doing that three years from now to 3 TBs of data will probably either need a lot of patience with a BD burner, or similar.

So hardware RAID sounds like the sensible way of allowing me to grow this without a massive pain.




Cool, I looked at the feature set and I'm convinced it can do all of that. Hardware transcoding using the ION graphics card I wasn't sure of. Being FreeBSD base will likely mean it's on the bottom of nVidia's todo, and the Atom is unlikely to cut it. I'll definitely install it and try it before I commit either way though.

So another attempt about what to purchase,

■A 4+ drive bay Micro ATX. I can comfortable fit 5 in the ones I've pasted earlier, maybe 6/7 with a bit of changes in the drive bays. Just need a bit more research before I pick one.
■Same POV motherboard. (Has PCI Express)
■Adaptec 3000 series. Probably 4 ports (3405 ) will be enough and use SAS expanders when the time comes to grow. (To use the PCI Express slot on the POV)
■Start with 3 x 1.5 TB disks RAID 5. Grow in 1.5 TB chunks in later years as required.
 

KuleRucket

Distinguished
Apr 4, 2009
4
0
18,510
I am so glad I found this thread. I am starting to look into my own NAS and many of my requirements are similar to yours. I have already been convinced by this thread that a Micro-ATX is the way to go and I've been focusing to much on the Mini-ITX. Those drives will need some air flow after all.

I had not considered a CF drive. I will have to do some more reading up on that I think.

One question I have though. I had already settled on using OpenSolaris as the OS with ZFS and RAID-Z due to the RAID-5 "write hole" flaw, but I noticed someone above mentioning ZFS on FreeNAS. I hadn't realised that FreeNAS supported these.

I have scoured the internet and there seem to be so many people looking for a 4 disk low power consumption 'open' NAS system based on the Atom or VIA Nano, but there are hardly any pre-built solutions. I think all the manufacturers are really missing a trick here.

Anyway, back to the research.....
 
I want to remind everyone who's looking at a NAS solution to remember that NAS is not a backup. If your data is important then it needs to be backed up to external media, and if the data is really important then one of the backup copies should be stored offsite.

So if you have a budget of "x" dollars, don't go planning on buying "x" dollars worth of NAS - you should really be thinking about putting some of that budget towards backup media as well.
 

afifim

Distinguished
Aug 19, 2009
4
0
18,510
BTW I ended up going for hardware raid 5 and that meant I ended up in a mid tower case.

Main consideration is the ability to grow the file system in a few years time and on line expansion for the Adaptec cards I am sure will be worth every pound when the time comes.

Hard drives were Samsung F2 eco drives 1.5 TBs. The lower spindle speed makes them quieter and more efficient with power.

Its 3 drives and I get a sustained 60 MB/s+, peak of around 90 from my Windows machine over the wire to it.

I did end up using PS3 Media server as well, it works a treat for playback on my Xbox 360, and PS3. I actually never spent time working out to get hardware offloaded transcoding to work (although I think ffmpeg supports it in Linux now which PMS uses), mainly because PS3 media servers transcoding ability was poor (if you try and browse chapters for e.g. it would generally time out before you get the thumbnails on the PS3 for e.g.).

This wasn't Atom being poor, its true on my i7 machine as well.

I didn't go for the CF card either, given my 1.5 TBs of disks cut my 1 TB hard disk out when it comes to hardware raid, I used that for the OS of the NAS device.

All in all I wouldn't fault anything, other than if a decent case came out that allowed a mini itx case with 5 drive bays, and a single PCI express card I'd be so there...

I don't get why there isn't such a product in the market place, because after all some of the "closed" NAS devices have some fantastic cases.

I can't measure the power draw of the system, but on idle I can't imagine it using more than 30 W, and when under load I can't see it using more than 70 W.

I went with Ubuntu in the end for ease of use, + slightly crude, I like having a Linux desktop around, freebsd just didn't tick my geekbox for me for some reason.

GL with your hardware selection. and as I said if you do come across a decent case please update the thread though as I did go for something quite cheap (like £25 mid tower) after spending like >£600 on RAID controller + disks + mobo, and I didn't want to shell out £100+ for a case that never ticked my Mini-ITX box.
 

sub mesa

Distinguished
Shame i didn't see this topic earlier, very interesting stuff. =)

ZFS allows growing striping volumes, but not parity volumes. You can always add several RAID-Z's to the same storage pool, so if you have 4 disks in RAID-Z (raid5) you can add another 4 disks and have two 4-disk raid5 arrays combined in one storage pool.

I have some questions though:

but given that I am running FreeNAS for my own NAS, I do not recommend the software RAID5 implementation built into FreeNAS.
Any reason you do not recommend it? It should prove a very fast RAID5 implementation, faster than hardware RAID in many cases, when properly configured. Though if i remember correctly, FreeNAS did use some older versions of the RAID drivers; don't know if that's fixed in current builds.

Me myself am running ZFS v13 on FreeBSD 8.0 (RC2), and it totally rocks. You do need 64-bit though, and recommended 4GB+ memory, i'm using 8GB at the moment, though i eventually want to upgrade even more. Memory quantity is very important for ZFS.

As for the 'NAS is not a backup' i'm not that sure, ZFS is no normal filesystem. It protects against filesystem corruption, repairs any damage on-the-fly, has resilience against file-deleting virusses or user-accidents due to copy-on-write model and countless other benefits that do not apply to conventional filesystems. Essentially, ZFS is a backup system, RAID engine and filesystem in one. Its still a single point of failure though, meaning that if ZFS contains bugs it may still fail even with many redundancy. Currently in FreeBSD 8.0 ZFS is not considered experimental anymore.

As for power: i do not recommend any Intel Atom or VIA Nano boards. They use even more power when idling than an IGP AMD based system; have only about 20% of the performance, is twice as expensive and often limited to 32-bit in case of the Atom; which severely limits your ability to run ZFS. For conventional RAID or hardware RAID the Atom is fine though, but not for ZFS.

Anyway you got your system already so i won't go into too much detail about a solution you didn't opt for. But if there are people who are interested in going this route i'm very motivated to help and guide them. :)
 
In principle NAS is no different that RAID in terms of suitability as a backup - that is to say: it's not. The problem is that there are lots of scenarios that it can't protect against:

- accidental deletion of files (depending the NAS configuration and which features the clients are capable of taking advantage of)
- theft / fire / other disaster
- file corruption due to software or virus problems on the client systems
- file corruption due to hardware / network / software issues on the client or NAS system
- power issues that fry your equipment

These types of issues are why good backup strategy includes offline and offsite media, neither of which a NAS provides on it's own.

If you have multi-terabytes of storage you may well decide that not all of the data is of sufficient value to offset the cost of backing up every last bit of it - and that's fine. But if you just blindly assume that you're covered because "it's a NAS!" then you're setting yourself up for disappointment.
 

sub mesa

Distinguished
Theft/fire/disasters: sure

But not accidental deletions/virusses etc. Even if the virus wipes all data or overwrites it badly, just one command to return everything to a version in the past. ZFS can do that, its so sleek. Its all thanks to copy-on-write. Essentially, when you change something, it'll never overwrite existing data; if you overwrite a file with totally different data the old version is still on the disk, and can be used to restore the entire filesystem to an earlier version with a single command.

So ZFS is also backup solution; it has a 'history'. It would offer the same protection as a backup solution physically close to the original data; won't protect against fire but will protect against various user/application issues.

Ofcourse, this argument fails if ZFS itself is bugged. Since its pretty new, any crucial data should of course be properly secured to another physical location and preferably also offline.
 
I'll admit that I am still using FreeNAS v.686 with FreeBSD 6.2. This version is a couple years old. Just never got around to upgrading my version of FreeNAS. But then again, that is the reason I chose FreeNAS, IT JUST WORKS! But I digress...

Given that I am using an older verison of FreeNAS, the RAID drivers are also not the latest. I found that the RAID5 solution in FreeNAS worked but had high cpu usage and being that I have a single core Celeron420 running my NAS, it actually bogged the cpu down. The cpu usage was high enough to effect data transfer across the LAN; so, I swapped out the software RAID5 for a 3Ware controller card. A dual core proc may help, but I'm not even sure if FreeNAS is mutli thread enabled; I'd be surprised if it was given how light it is.

The next project on deck is to update/upgrade my NAS and with it move to the latest version (.7RC1) which upgrades the FreeBSD kernel to 7.2 and includes ZFS support. Looking forward to playing with ZFS; perhaps the software RAID solution has been improved as well.
 

sub mesa

Distinguished
FreeBSD 7.2 / FreeNAS 0.7 still has the 'older' version of ZFS, or ZFS v6, while FreeBSD 8.0 has the latest version of ZFS, v13. Only v13 is considered stable and has the 'experimental' tag removed. Its also highly recommended to run 64-bit distribution when using ZFS.

As for the RAID5 driver; doesn't help if you're gonna run with a Celeron, with an FSB instead of IMC. You need memory bandwidth and FSB-based cpu's will simply suck at this type of workload. Dualcore is pretty much mandatory too, though its not fully threaded FreeBSD itself is highly scalable in SMP workloads. Much better than Windows. In fact both Linux kernel and the MySQL project benefit from FreeBSD innovations in SMP performance.
 
G

Guest

Guest


Can you please share what you think is a good solution, thanks.
 

sub mesa

Distinguished
As for hardware? Well as i said a simple AMD IGP system would consume very little power. IGP means onboard graphics (Integrated Graphics Processor).

Quoting a post from this thread:
Low-power means something like 35-50W excluding disks. This is easily achieveable with an AMD setup and efficient power supply.

For example:
■ AMD Dualcore with 35W or 45W tdp means ~2-3W idle
■ AMD 740G/780G/785G chipset means ~2W idle
■ Amacrox 400W fanless PSU is 90%+ efficient, one of the most power efficient power supplies and comes without a fan

If you go for 2,5" HDDs you can even go for an "extreme" setup with PicoPSU power supply, which is like ~97% efficient and can reduce idle power consumption to 30-35W, while still providing high performance with lots of memory and dualcore chip to saturate your gigabit NIC.

Note that TDP can be misleading; while the maximum power consumption is important for chosing the correct power supply, its not important for actual power consumption, as your hardware will idle most of the time. In fact, all PCs in the world idle more than 90% of the time. Idle power consumption is what you should look at if you are concerned with the environment and/or want to save on energy costs.

For example i'm using the AMD Athlon 64 3800+ EE SFF 35W TDP dualcore cpu at 2.0GHz, but its very rare and hard to get. But other 45W TDP cpus can use just as much when idling. the "e" series stand for energy efficient, for example the AMD Athlon II X3 400e series is a triplecore with 45W TDP thats reasonably low on idle power but packs alot more power than an Intel Atom cpu.

Intel Atom is power efficient, for sure, but since its paired with very unefficient chipsets, the benefit is negated and it consumes as much power as a setup i described which has much much much more processing power.

Here is another thread with some useful information about ZFS:
http://www.tomshardware.co.uk/forum/page-253172_14_0.html
 
G

Guest

Guest
Thanks for the reply, very useful, I've only just started looking into the parts needed for my NAS.

My first rev. list comprised of:
CPU: SDX140HBGQBOX
Lian Li PC-A06F case
Hot swap/caddys for the HDDs: e.g. Icy Dock MB-455SPF 5x3.5" Sata II Hot Swap Aluminium Backplane into 3x5.25" Bays
Motherboard: Gigabyte 785G motherboard (not sure which yet)
Maybe: Highpoint raid controller
PSU: Corsair 450W 85% efficiency CMPSU-450VXUK
HDDs: minimum 3x 1.5TB Samsung EcoGreen F2 SATA-2 Hard Drive in RAID 5, maybe a couple of something smaller for the OS.

I'll look into the CPUs as it may not hurt to have something with a bit more grunt as you suggest.
 
G

Guest

Guest
Have spent some more time looking into things and am now looking at the following hardware, any thoughts?

Scan.co.uk
400W Corsair Power Series PSU, ATX, PS/2, UK Version 3 year warranty (80+ Certified) LN24874 £36.78
Gigabyte GA-MA785GM-UD2H, AMD 785G, AM3, DDR2 800/1066/1200, SATA 3Gb/s RAID, Micro ATX, VGA LN28390 £61.30
AD235EHDQBOX - AMD Athlon II X2 235e Energy Efficient Dual Core, S AM3, 2.7GHz, 2MB Cache, 45W, Retail LN29656 £53.75
TWIN2X4096-8500C5C - 4GB (2x2GB) Corsair TwinX DDR2 XMS2, PC2-8500 (1066), 240 Pin, Non-ECC Unbuffered, CAS 5-6-6-18 LN27732 £78.94
1.5TB Samsung HD154UI Spinpoint F2 DT EcoGreen, SATA 3Gb/s, 32MB Cache, 8.9 ms, NCQ LN27763 £71.84
1.5TB Samsung HD154UI Spinpoint F2 DT EcoGreen, SATA 3Gb/s, 32MB Cache, 8.9 ms, NCQ LN27763 £71.84
Samsung SH-S222A/BEBE 22x DVD±R, 12x DVD±DL, DVD+RW x8/-RW x6, DVD-RAM x12, IDE Black OEM LN25351 £17.22
43cm Scan Serial ATA Cable SATAII (RB-404) LN6782 £0.60
43cm Scan Serial ATA Cable SATAII (RB-404) LN6782 £0.60
43cm Scan Serial ATA Cable SATAII (RB-404) LN6782 £0.60
43cm Scan Serial ATA Cable SATAII (RB-404) LN6782 £0.60
Overclockers.co.uk
Fractal Design Define R2 Midi Tower Case - Black Pearl (No PSU) CA-003-FD £89.98
Samsung EcoGreen F2 1.5TB SATA-II 32MB Cache - OEM (HD154UI) HD-080-SA £72.99
Samsung EcoGreen F2 1.5TB SATA-II 32MB Cache - OEM (HD154UI) HD-080-SA £72.99
£630.03