Sign in with
Sign up | Sign in
Your question

Build a Cheap and Fast RAID 5 NAS

Tags:
  • Article Discussion
  • NAS / RAID
  • Networking
  • Build
  • Product
Last response: in Toms Network
Share
August 1, 2006 3:30:11 PM

Can a RAID 5 NAS that you make yourself outperform ready-made alternatives? Bill Meade finds out.

Speak out in the Tom's Networking reader survey!

More about : build cheap fast raid nas

August 1, 2006 6:13:06 PM

Nice review :) 

I have now since about a year something comparable:
- athlon xp 2200+
- MSI Athlon MP server board
- 1GB registered ddr ram
- 3Ware 7506 raid controller
- 4x WD hard drives 200GB
- Intel 1000GT network card

I use Fedora Core 4 on it combination with Samba, Apache and MySQL.

The average performance mentioned in the review is about the same in my case. Both reading and writing peaks at 40MB/s

I saw in the Samba configuration also a home directory share. There's something more usefull with Samba. It's possible to set all home directories shared automatically (no matter when a user is added/deleted/..).

You need the following configuration into the share section:

[homes]
read only = No
browseable = No

I'm not sure if this will work in share level security (I use user level). But anyway, with the right chmod settings per home directory, your able to limit every user to it's own home directory (in case you log on on the system itself and in case of accessing the shares through the network). The logged-on user in the network will also only see his/her share in the Network Neighbourhood (Windows).
August 1, 2006 7:05:26 PM

Edward said:
Quote:
Bill,

I just read your article about the Raid 5 storage and I had a quick question. how much effect does the processor speed have on this machine. I always figured that it did not. How do you think an old Pentium II would do in a similar configuation? How about an old athalon 1GHz?

Thanks for your time, I enjoyed reading it.
Related resources
August 1, 2006 7:06:55 PM

John said:
Quote:
Sounds like a nice project. One thing I have learned over the years is the cheap 10/10/1000 adapters sold just are not worth it though. The OEM edition of the Intel Desktop adapter uses a FAR superior chipset to anyone else's and can be had for about $30 as seen here http://www.newegg.com/Product/Product.asp?Item=N82E1683... or the retail version for about $3 more. It is definitely worth it.

Processor load is far less with Intel cards than with cheaper ones. I have not seen a test with 3com or other higher end products but otherwise, I would spend the extra $15 to make sure this system delivers its peak performance. Utilizing 'nix OS is often far more effecient than Windows but the Intel card will make this thing scream. Another option is purchasing a motherboard with the nforce 10/100/1000 on board. They are often fairly high performance/low CPU usage as well.
August 1, 2006 8:07:31 PM

I built my NAS with the 6/06 release of Solaris 10 utilizing ZFS.
Intel D820, Asus P5LD2 mobo, 1GB ram, onboard sata controllers, 250GB sata drives(4) in raidz, netgear 1Gbit nic.

With it, I get average speeds of ~38MB/s read, ~29MB/s write over nfs.
I haven't run iometer tests on it yet, these are just calcs with dd from a remote linux box.
August 1, 2006 8:41:15 PM

A good article on building a cheap RAID 5 box, but I have to disagree with your selection of RAID card.

If you build everything from the ground up and want to save money, then by all means, go with the hardware mentioned in the review.

Many folks, though, will not be starting from scratch. Most people who will take the time to read this article and tackle the project will most likely have a fair amount of equipment already on hand. If all you need to buy is the card and the drives (assuming you have everything else), then you definitely should go with a SATA version. I use, and recommend, the Areca PCI-E SATA II cards. Although they are a little pricey, they definitely give you a long term usage strategy. PCI-E and SATA are here to stay for quite a while, so buying one of these cards now will stave off obsolescence for the foreseeable future. You should be able to get a good Areca 8-port card and four 300GB drives for the same bottom line price as all the equipment listed in the article. This leaves you 4 free SATA ports to add storage to your array when you need it.
August 1, 2006 8:53:17 PM

Quote:
Edward said:
Bill,

I just read your article about the Raid 5 storage and I had a quick question. how much effect does the processor speed have on this machine. I always figured that it did not. How do you think an old Pentium II would do in a similar configuation? How about an old athalon 1GHz?

Thanks for your time, I enjoyed reading it.


Edward,

I'll be able to tell you more in a couple days. I'm re-running the tests on a 1.6 Ghz Duron right now. I don't have comparable numbers (I'm adding Windows XP to the operating systems, but even XP is surprisingly fast without RAID). My experience with Pentium II and Pentium III is that I am too pessimistic. I've set up ClarkConnect home servers for friends expecting them to want to upgrade. They never do!

So, try it, you will probably not care. The Linux big file problem is the bottleneck in these results, not the CPU I suspect.

bill
August 1, 2006 9:08:53 PM

Quote:
I built my NAS with the 6/06 release of Solaris 10 utilizing ZFS.
Intel D820, Asus P5LD2 mobo, 1GB ram, onboard sata controllers, 250GB sata drives(4) in raidz, netgear 1Gbit nic.

With it, I get average speeds of ~38MB/s read, ~29MB/s write over nfs.
I haven't run iometer tests on it yet, these are just calcs with dd from a remote linux box.


krisia2006

The iozone benchmark is pretty easy to set up. Here is an example command line. When you install iozone it wants by default, to live in /Program Files/benchmarks/Iozone 3.326. So you run a dos window, switch to that directory, and then you feed iozone a command like this:

iozone -Rab 014XPD16.wks -i 0 -i 1 -+u -f z:\test\013.tmp -q 64k -n 32M -g 1G -z

The above is a command I just ran testing Windows XP on a Duron 16. I like to code what the hardware is into the output .wks file name. Iozone runs the tests and lists the data on screen and then dumps everything into the .wks file. So, it is really pretty easy to replicate what Tim Higgins has set up. If I can do it, someone who can touch solaris and live can do it easily! :-)

Tim's how-we-test article is available at http://www.tomsnetworking.com/2004/03/02/how_we_test/

If you want any help I'd be happy to step you through it.

bill
August 1, 2006 10:02:09 PM

Quote:
John said:
Sounds like a nice project. One thing I have learned over the years is the cheap 10/10/1000 adapters sold just are not worth it though. The OEM edition of the Intel Desktop adapter uses a FAR superior chipset to anyone else's and can be had for about $30 as seen here http://www.newegg.com/Product/Product.asp?Item=N82E1683... or the retail version for about $3 more. It is definitely worth it.

Processor load is far less with Intel cards than with cheaper ones. I have not seen a test with 3com or other higher end products but otherwise, I would spend the extra $15 to make sure this system delivers its peak performance. Utilizing 'nix OS is often far more effecient than Windows but the Intel card will make this thing scream. Another option is purchasing a motherboard with the nforce 10/100/1000 on board. They are often fairly high performance/low CPU usage as well.


This is really great information thanks for posting it. It is one thing to suspect that some component is better than another. But it is another to have expeience with it. I had a conversation with the ClarkConnect president and his chief hardware guru about ethernet adapters and their preference was 3com. Of course, the 3com adapters are PCI-X (64 bit) and all the motherboards I've been playing with are 32bit. Then intel you mentioned is 32bit so it is definitely worth a try. Have you had any bad luck with open box products from NewEgg? I've gotten close to buying them in the past but NewEgg's warnings are so draconian I always chicken out!

Thanks!

bill
August 1, 2006 10:13:50 PM

Quote:
A good article on building a cheap RAID 5 box, but I have to disagree with your selection of RAID card.

If you build everything from the ground up and want to save money, then by all means, go with the hardware mentioned in the review.

Many folks, though, will not be starting from scratch. Most people who will take the time to read this article and tackle the project will most likely have a fair amount of equipment already on hand. If all you need to buy is the card and the drives (assuming you have everything else), then you definitely should go with a SATA version. I use, and recommend, the Areca PCI-E SATA II cards. Although they are a little pricey, they definitely give you a long term usage strategy. PCI-E and SATA are here to stay for quite a while, so buying one of these cards now will stave off obsolescence for the foreseeable future. You should be able to get a good Areca 8-port card and four 300GB drives for the same bottom line price as all the equipment listed in the article. This leaves you 4 free SATA ports to add storage to your array when you need it.


All I can say is Doh!

One of the bad things about having a RAID habit, is that you don't often run into people you can talk about RAID with. Then, when you do run across someone, as you ran across my article, they are interested in some different aspect of RAID than you are.

My bad. In my article, the whole idea was how much performance can I get with hardware that is the minimum configuration to get the job done. "How little can I get away with?" is my big question. I didn't consider your RAID card because I have been living 100% in the used RAID card market!

Your thing is clearly performance. I can dig that too. Just not while working for Tim Higgins :-). Reading about RAID the disk reads should be faster. The writes should be slower. The relationships between all the behaviors are described theoretically. We need to test these because a lot of the time, the theories are wrong.

Do your SATA drives have command queueing? How does command queueing change RAID behavior? It would not surprise me if command queueing tripps up the striping that the RAID adapters are trying to do. And, what about defraggin RAID arrays? Is that a good thing, neutral, or a bad thing?

Thanks for the recommendation! One thought I've had is to take a great RAID card, like yours, and re-do the analysis with another slow CPU. I was hoping to use the 4 WD ATA drives, but I may have to spring for new SATA drives as well. No .... maybe I can us one of those ATA to SATA adapter kits!

Just dinging you! Thanks again!!!

bill meade
August 1, 2006 11:07:35 PM

A small point i wanted to make - living in the UK, or computer equipment is a bit more expensive than the US.

I tried to configure a BYO system with similar kit - slightly higher end, e.g. a case with decent cooling, a good psu, but definitely sticking to budget stuff (about the 2nd cheapest model I could find).

I found the cost close to ready made solutions like the Intel SS4000-E.
Given that it's ready made, with more advanced features and needs little configuration, I'm inclined to think that it only makes sense to build your own if you (a) have most of the kit lying around (perfectly sensible) and (b) have the time to do it.


I do NOT mean to suggest in any way that the article wasn't excellent, by the way. I hope there will be many more like it!


One weakness of the ready made solutions though is that they all seem to have annoying bugs, e.g. Buffalo Terrastation doesn't support very long filenames, the Intel SS4000-E, has a lot of bugs, doesn't support NTFS, and while its the fastest of the existing equipment its still way slower than it should be.

One area that it would be interesting to see, for home/SoHo users is to build a RAID inside a workstation, using on of the new motherboards that support up to 10 SATA devices. You could easily allocate 4 of them to your RAID setup for ackups etc, keeping the rest for everyday use. Apparrently the performance of the built in RAID controllers is quite good. And of course you could still use the Gigabit interface to connect to other clients.

Any thoughts?
August 2, 2006 12:13:10 AM

Quote:
It's essential to understand that the system reviewed has a bottleneck in the old PCI bus i.e. 32 bits @ 33 MHz <= 133 MB/s aka ATA-133 MAXIMUM (that's where the "133" in ATA-133 comes from.


That was an ATA100 controller... :wink: besides find a single drive that can sustain 133MBs, let alone burst a read at the speed.

Explained:

Congestion on the host bus to which the ATA controller is attached may also limit the maximum burst transfer rate. For example, the maximum data transfer rate for conventional PCI bus is 133 MB/s, and this is shared among all active devices on the bus.

In addition, as of October 2005 no ATA (this includes SATA) hard drives exist capable of measured sustained transfer rates of above 80 MB/s. Furthermore, sustained transfer rate tests do not give realistic throughput expectations for most workloads: They use I/O loads specifically designed to encounter almost no delays from seek time or rotational latency. Hard drive performance under most workloads is limited first and second by those two factors; the transfer rate on the bus is a distant third in importance. Therefore, transfer speed limits above 66 MB/s only really affect performance when the hard drive can satisfy all I/O requests by reading from its internal cache.


Most would soil themselves if they had an array that could sustain that speed in a DIY “cheap” box.

Quote:
And, certain Areca versions can be upgraded with as much as 1 GB of on-board cache, for real I/O acceleration (might be a special order from Areca, or so our systems consultant tells us).


This is about "cheap" NAS.

Quote:
The Promise model EX8350 is very similar in overall
design and performance, and a bit less expensive.
This is an x4 lane device.


$400-$1000 cards aren't what I would call cheap. If they are pick me up a few at the corner mart and a 20oz drink as well. While you’re at it I’ll take an 2105 ESS made by IBM as well.

I would say four drives stays within the bounds of cheap. An $80 card still stays in the cheap zone. I’m setting $500 and below as the standard and quintessential precedence for “cheap.”

Bill maybe you could run those numbers on raids, 0, 1, 0+1, 10, and 100 for those who like me don't like Raid 5, but I do agree 5 is really easy and safe.
August 2, 2006 1:18:55 AM

My thoughts:

First, "Amen" to getting the best stuff for the least. For every job there is a perfect tool. If you can get a cheap perfect tool. Buy it! I don't mean to advocate that people build PCs from scratch for this project. I did not do that. I took an overly expensive power supply I ALREADY HAD, a cheap mobo I Already Had, memory, you get the idea. I even had the RAID controller because I had bought 5 of them at one time to get the price down. Possession takes the *sting* out of acquisition!

Second, the ugly tradeoffs of RAID are not a product of commercial vs. home-built systems. They are a function of RAID's immaturity in the personal computer ecology. File naming is a problem. I'm a Mac user and this is always tripping up clean backups. What to do .. step-by-step to recover from a bad hard drive is a problem. Which RAID to use is a problem. RAID is the "media PC entering-the-we-are-not-proud-zone" of the storage world. All potential, weld splatter, and rough edge. THE BIGGEST UGLY THING is the large file slow down of Linux-based RAID systems. When you get a file larger than 600 megabytes, Linux grinds to a halt. 2.4 kernel, 2.6 kernel, makes no difference. LINUX SUCKS ON LARGE FILES. I've never heard anyone say "Oh, that does not matter, we don't need to fix it." but it sure seems like key people in Linux land are thinking that to themselves.

Third "Amen" to a RAID rack whether it is inside or outside a workstation. I love the RocketRaid external enclosure, the X4 which is $450 http://www.newegg.com/Product/Product.asp?Item=N82E1681... and connects to the internal RocketRaid card via Infiniband.

"Infiniband" what a cool name, makes me want to flush money down the toilet NOT!. But it is a cool name. This product if it housed 10 drives would be VERY COOL because I don't know how many 160 gig hard drives you have, but I have like 10 of them. They'd make a nice 1.6 terabyte array given the right hardware.

My other thought on this is that I know way too many people who have discovered that firewire is unreliable. Wiebetech is living off firewire 800 direct attached storage systems that are way fast. But I'm skeptical of putting my life on firewire. It would be nice to have some other more robust standard like a dumbed down fibrechannel. Doh, probably what Infiniband is.

Look at the sustained transfer rates they are getting from RAID0 sata drives. Link: http://www.wiebetech.com/products/G5Jamplus.php I've been asking Jeff Hedlesky (Wiebe Sales Guru) for (a couple) weeks to loan me one of these setups so I can see if it is real or if it is Kool-aid. Yo Jeff, got your ears on????

Dimitrik you raise a number of great points. What I think they all point to is that we are just beginning to figure out what RAID can do for us in the way of reliability and performance. And ... what RAID can do for us in the area of re-using or repurposing computer resources like drives and NASes.

Thank you for posting. If I missed "ask again" and I'll try again.

bill
August 2, 2006 1:27:35 AM

I'm re-running the numbers as I write this, on non-RAID NAS configurations. I can't seem to trick FreeNAS into working on hardware without the LSI Logic i4 card in it (did you notice we snuck FreeNAS into the graphs?). But I'm testing Win XP, ClarkConnect Home 3.2, Ubuntu Desktop 6.06 on dog-slow (Duron 1.6 ghz) hardware. I'm surprised by how the good the results are.

Makes me want to find a more current RAID card and then try RAID 0, 1. I ran RAID 10 for a year and couldn't stand the wasted drive space. What is 0+1? I haven't heard of 100. Can you please reply and educate?

Thanks!

bill meade
August 2, 2006 1:31:23 AM

I did this same thing 2 and a half years ago. My approach was to compare software raid with hardware raid using SATA technology. I picked up a sata version LSI controller and 5 hitachi 250GB disks, etc. There are some nice hot-swap cages that improve airflow over the drives from enlight and others that simplify this too, for 2, 3, 4, and 5 drive bays. Using the hardware based raid I was actually depressed by the performance. Somewhat lower than your numbers, around ~5000kB/s sustained, but I was learning. Moving to software raid on linux I was able to sustain 7-12kB/s per drive. IOStat is a great tool that isn't normally installed in most distributions.

You are wise to establish how to repair the raid when it breaks before starting. Assuming you are using MD for your software raid, its actually quite simple to repair using mdadm commands. Insert the new drive, add it to the raid group and let it rebuild. Its a good idea to remove the faulty drive from the configuration first. I avoid lvm on linux as it incurs roughly a 33-50% preformance hit. It has nice features but the cpu spends too much time at idle or IO wait states.

to fix md raid 5 is something like this (assuming you've already swapped disks)
mdadm --manage /dev/md0 --remove /dev/sda (something like that)
mdadm --manage /dev/md0 --add /dev/sda
(*I don't know if thats accurate syntax - thats why there are manpages)
Watch /proc/mdstat for progress (cat /proc/mdstat)

I've recently upgraded my nas to the following - (don't cry)
AMD64 x2 3800, 1GB DDR2(800), 1GB nic, linuxfromscratch.org OS. Still running MD for raid5 - I've only done rough performance numbers of 90MB/s across the GiGe port sustained for 4 minutes.

I feel there are two benefits to software raid - memory or cache is important - linux will use available memory as cache for the filesystem and that can increase perceived performance. The inexpensive hardware raid adapters don't have enough IMO. Secondly, accessing the disks. All raid5 configurations write the data in a proprietary format - accessing this data after a crash can be difficult, especially with hardware raid, but software raid too. I feel more comfortable with a linux/software raid knowing that I can recover my data manually. However I have a few advantages - I built it, I've worked with linux on and off for 13 years, and I work in the storage industry, mostly with fibre channel.

Some tips I picked up---

Someone asked what impact the processor will have, really it all depends. I started on a Via 500mhz itx board with 512MB memory. The cpu's main task is to calculate the parity in a raid5. Hardware raid should handle this so the cpu becomes less important. The more memory you can add to the raid solution helps also. Most enterprise storage arrays have gigabytes of cache memory. A 64bit OS can increase performance also, so the newer AMD64 and intel 64 chips will help, but you don't need the fastest one out there.

Power - Disk drives consume roughly 2 amps of power to boot or spin up and can use the full 2a under heavy conditions. Verify your powersupply can handle the total number of amps on the +12v line. So 3 disks at 2a each would be 6a total. Don't forget the internal one plus the CD/DVD? Each vendor lists their drive's consumption on the web.

Samba - You could remove the force user and force group and go with only guest only = yes or allow guest = yes. Remember to chmod 777 your shares also! :o ) User level security requires authentication, share level is dependant on the share itself, there is also Wins, domain, and ads options, good for keeping the kids or parents out of your stuff!

Great article, nice to see others doing the same geeky stuff I'm doing.
August 2, 2006 1:36:09 AM

If I recall, raid 0+1 is where you stripe first, then mirror the drives. The performance of raid 0 with the (theoretical)reliability of mirroring.

Hey - who's handing out 2105s? I'll take one too! :o )
August 2, 2006 2:09:49 AM

So:

RAID10 is RAID1 + RAID0? (Parity + Stripe)

4 drives and you get 1/2 their capacity and (theoretically) the drives are fast.

0+1 =RAID0+RAID1 (Stripe + Parity)

4 drives and you get 1/2 their capacity and (theoretically) the drives are fast.

When do you use RAID10 and when do you use RAID0+1? My MANAGING RAID ON LINUX by Vadala 2003 is getting a little dated! It mentions RAID0, 1, 10, 2 and 3 (both obsolete), 4 (dedicated parity), 5 and RAID50 (striped parity). Is RAID50 (striped parity) the same as RAID0+1?

Does anyone else ever feel like they are beginning to become certain they know how many angels REALLY fit on the head of a pin? :-)

bill
August 2, 2006 4:07:58 AM

Quote:

One weakness of the ready made solutions though is that they all seem to have annoying bugs, e.g. Buffalo Terrastation doesn't support very long filenames, the Intel SS4000-E, has a lot of bugs, doesn't support NTFS, and while its the fastest of the existing equipment its still way slower than it should be.


About 3 years ago, I build a RAID 5 server to hold my digital media for about $1000. I was putting all my CDs into digital format, and quickly filled up my 250 gig hard drive. I wanted something safe, and something networked. I build a server around a Pentium 4 board, with an Adaptec 2400a card. It worked well.

However, there were a number of problems with it. First, I didn't need it on all the time, just when I was in the studio. I couldn't figure out a way to turn it on at set times. That was a pain. Second, it was loud. 4 250 gig drives (5400 RPM), plus a system disk. It was loud. Third, it generated a LOT of heat. Fourth, there was no way to expand the array. I never thought I'd need more than 700 Gigs of space. I was wrong. The Adaptec card does not support expansion.

I replaced it with a Infrant ReadyNAS NV. The thing uses less electricity, is 5 times quieter, can turn itself on and off, supports SATA drives, can expand the array, works with long file names, and it's 8”H x 5”W x 9”D. It's amazing. I looked at the Buffalo, and the Intel. The Infrant hands down beat them, and their support is amazing.

I'd never think about doing the geek thing and making my own unless I have a closet with it's own AC and lots of noise damping.
August 2, 2006 5:01:54 AM

Interesting article, but i'd take issue with your design for one major reason.

You state you are building a NAS - this is an APPLIANCE SERVER. It should not be expected to do anything other than be a device for storing data. Therefore you should have gone for a fairly low end CPU (Semprons and Pentium M's are great for this sort of thing), Software RAID (based either on Windows or Linux) and a good whack of RAM (1 GB at least). The NAS could then churn over its fairly minor processing tasks of calculating parity values all day whilst boosting performance using its large RAM as a data cache. Rebuilding RAID 5 is simplicity itself under both Windows Server (or Hacked XP) and Linux, it only takes a few mins to learn.

I think what you ended up building is a regular file server - using a hardware raid controller is only necessary if you need to offset the processor load which in a NAS you dont. The thing should never do anything but worry about RAID and maybe run backups.
August 2, 2006 5:16:36 AM

Those are excellent points minty - I admit to using mine as a full-fledged server with vnc so I utilize the processor ripping video formats.
August 2, 2006 7:41:42 AM

It would be a shame to spend that much money on hardware RAID, do that much work, and leave so much performance on the table.

A Windows folder share on GbE easily does file copies at 35 MB/s. No reason to build a NAS system without such capability. Motherboards with GbE are cheap and a CPU to handle the load would be worth the extra money.

I think Thecus just came out with Celeron based NAS box that sets the bar for any homebuilt system to match.
August 2, 2006 8:38:49 AM

I'll add my two cents:

I have three systems setup with RAID 5, one has 4 320Gb WD's on a highpoint 2320 pci-express 4x card on winxp pro 64bit. The 2nd has 8 HD's over 4 controllers (2 onboard, 2 addin sata and ide) doing software RAID 5 on windows xp pro, the 3rd has 4 250GB WD RE on a highpoint 1640 with server 2k3.

When it comes to transfering data to any of them, the 2320 is fastest, software on xp is 2nd and 1640 is last, all with gigabit connections. They are all fast enough to move amazing amounts of tv shows, movies, music, etc, in very little time and I honestly can't notice a difference in speeds in real world use.

Benchmarks show the 2320 about 20mb faster than the other two on reads and 14mb faster on writes. We're talking around 80-110 on write and 110-160 on reads with the 2320 bursting over 200. The 8 drive system actually bursts around 140-150 as it's not limited so much to the pci bus since it's using onboard controllers while the 1640 stops dead at 110.

All three have been very reliable, not 1 drive failure in the 8 months I've had them all running. The array with 8 hd's is made of random drives I've had laying around, some as old as 5 years.

The 1st system is an athlon 64 3700 with a gig of ram, 2nd is an old p4 2Ghz with 1gig ram, and the 3rd is an althon xp 1800 with 1 gig ram.

Everyone says software RAID is slower, maybe, but not noticably if you have a fast enough system, and a p4 2ghz isn't exactly fast by todays standards ;) 

I've got almost 3TB on these three systems and I use them all constantly, I would never trust this data to anything but RAID 5, except RAID 6 but I can't afford that :) 

Edit: I forgot to mention, I retain full NTFS permissions across my data servers since they all are joined to my win2k3 Domain Controllers, and even if it was a non-domain environment I wouldn't need to bother with samba issues. I should mention I have a couple of linux boxes and samba works perfectly to conect to the data servers with no trouble.
August 2, 2006 8:48:36 AM

Quote:

...
I replaced it with a Infrant ReadyNAS NV. The thing uses less electricity, is 5 times quieter, can turn itself on and off, supports SATA drives, can expand the array, works with long file names, and it's 8”H x 5”W x 9”D. It's amazing. I looked at the Buffalo, and the Intel. The Infrant hands down beat them, and their support is amazing.
...


Thanks, that sounds very interesting. I've just read the review on the site and it looks like a great piece of equipment. I'll have to compare that to building a windows based RAID for backups using an integrated RAID controller & GigE.
August 2, 2006 11:09:32 AM

Quote:
Can a RAID 5 NAS that you make yourself outperform ready-made alternatives? Bill Meade finds out.

As mentioned above, this system seems to be a little middle-of-the-road; ideal for neither of the tasks it's performing. Surely, either upgrade the processor in order to outsource heavy processing tasks, or use a VIA C3-based system to save space, noise & power...
August 2, 2006 11:33:36 AM

Quick RAID question for anyone who can help...

Does RAID 5 or any other from of inexpensive RAID work with drives that are of different size without wasting space? By wasting space I mean I know that if I partition both an 80 GB drive and a 100 GB drive each with an 80 GB partition, then I can mirror the two 80 GB partitions but I still end up with 20 GB not used in the RAID.

I'm in college so I do not have the money to shell out for several drives at once. Usually when I do buy a new hard drive, it is because I find them on sale somewhere and I am limited to buying only one. Thus, I have 3 x 80GB, 1 x 100GB, 1 x 120GB, 2 x 160GB, and 1 x 200GB drives to use in my backup machine.
August 2, 2006 12:00:30 PM

If you don't want to waste storage, all drives have to be of the same size.
August 2, 2006 12:11:42 PM

Nice article, might set-up a thing like this in my house (4 students).
I was thinking abou this bit though:
Quote:
The nightmare scenario here is having a teenager in a fit of rage erase everything in a share or two. Although RAID 5 is cool, it won't protect against that, so it is not sufficient as a stand-alone backup strategy...

With the ability to customize linux, wouldn't it be possible to put a delete-delay in? The raging kid presses delete, and it looks like it works. But in stead the files are just hidden and the actual deletion takes place say 24 hours later. In that 24 hours you can 'rescue' the files again.

Unfortunately the only programming I'm capable of is a bit of Java but I imagine this wouldn't be too hard to do for someone skilled :) 

Quote:
Thus, I have 3 x 80GB, 1 x 100GB, 1 x 120GB, 2 x 160GB, and 1 x 200GB drives to use in my backup machine.

Sell the smaller ones and with that money buy an extra big one or two ;)  Don't know where you live but in the US you may even find identical hard drives (from different sources) second hand.
August 2, 2006 1:07:37 PM

pah! who's idea was this article?? My xbox would beat your RAID 5 box any day ;-)

Seriously though, great article, exactly what was required - to show the gigabit consumer NAS boxes to be the frauds they are in terms of speed!

I dont think you mentioned what filesystem (EXT 3 etc) you formatted the drives with. Some of the threads mention that Linux is rubbish at handling big files, but I remember there being a filesystem specifically for large files thats used with MythTV etc. Perhaps that would be worth a try? Apologies if I missed it in the article.

nice one!
August 2, 2006 1:19:00 PM

Quote:
If I recall, raid 0+1 is where you stripe first, then mirror the drives. The performance of raid 0 with the (theoretical)reliability of mirroring.

I've found this page to be a good RAID reference:
http://www.acnc.com/04_01_0_1.html
August 2, 2006 2:01:14 PM

A couple of points you might consider/ criticisms and oversights in your review.

1\. Why not use a motherboard with built in RAID 5 (preferably sata sometimes even with RAID6) and a gigabyte Network port- as this would surely bring down the cost as most modern motherboards have them (by reducing the part count).

2\. My main concern with these setups is how much noise they make (especially since many such devices will be used to stream media), so investing in a quiet power supply, cpu fan would seem essential. It would also be usefull to put up benchmarks (subjective or otherwise)as to how quiet the system is against other off the shelf examples.

3\. If it is really to be an apples to apples comparrison, why not choose a sata setup to compete with the THECUS NVRAID system (which is better performance wise than the buffallo system).

4\. The power draw of a system is very important (especially is it going to be an always on system) so I wouldn't want my raid box to be consuming the eqivalent of 5 lightbulbs for occasional file server use. Here again benchmarks of equivalent power draw would be usefull. A smaller power supply would possibly be more appropriate (as an example a mini-itx format Power Supply with power connector splitters for all the drives).

5\. The physical size of the unit is also a consideration (do you really want a hulking great case in the middle of your house). For instance, including other cheap small footprint cases would have been usefull in your review.

6\. To compare it fairly with other off the shelf RAID systems out there, I feel that the ommission of removeable caddies was an oversight (especially against the Thecus products).

7\. Another important factor overlooked was boot time/ setup time (how long does the system take to power up and be useable over a network from an off state. Also how long does it take to rebuild an array after a disk failure/ replacement. Benchmarks like these would be very usefull.

8\. Although FreeNAS was dismissed, I feel it was overlooked in 1 critical area- that it can be booted from a USB key. IT is a more efficient system in terms of loaded modules (do youy really want a noveice to work out exactly which packages are necessary from an Ubuntu installation package? -after all this chews up valuable disk space).

9\. There was also no mention of how easy it is to set up in a headless network state (pretty easy in FreeNAS). There was also no mention of the lack of information/ control panel on the front of the case (that most off the shelf NAS products have),

In my opinion, I would not rate this as an effective comparison system. It is probably a reasonable how to article (but litte more) as it fails to address the key areas outlined above. It is the equivalent of showing you how to build a motorised go-cart (with a off the shelf parts) and then comparing it to a car. :o  :o 
August 2, 2006 2:02:24 PM

Quote:

...
I dont think you mentioned what filesystem (EXT 3 etc) you formatted the drives with. Some of the threads mention that Linux is rubbish at handling big files, but I remember there being a filesystem specifically for large files thats used with MythTV etc. Perhaps that would be worth a try? Apologies if I missed it in the article.
...


This is a really important question.

I think for windows users, NTFS on a windows RAID box may the easiest/safest option, however many of the pro-sumer RAID kit use Linux/Etx3 (very sensibly).

If you plan to use this as a media storage server for video etc, as many home users will, the filesystem has to be good at handling large files.

So the question about Ext3/large files becomes key.
August 2, 2006 2:35:29 PM

Interesting article. Just one comment, I'm not sure if, for the price of this solution is really worth the trouble to build your own. I was looking for a RAID solution for home a few months ago and I found a company called Infrant Technologies that makes NAS devices, I dediced to purchase a ReadyNAS RNX6 diskless plus 4 x 250GB SATA drives for a total $950 USD, which is almost the cost if this BYO solution in an already prepackaged unit.

I love those ReadyNAS units, specially their RAID-X technology, which main feature is to allow you to replace your disks with bigger capacity ones without having to backup/restore your data. And they are very easy to use via a web interface.
August 2, 2006 3:37:19 PM

For every job there is a perfect tool. I'm looking for an excuse to try the Enfrant ReadyNAS myself. But I have to ask you ...

what did you try to configure your home-built to wake from sleep?
IN THEORY we should be able to make our NAS boxes do this with power management and wake on LAN. But, I have never done so. My home NAS is on my mail server/router/web server. So the box is up all the time.

On expanding the array, the LSI MegaRAID i4 allows drives to be swapped in that are bigger. Once you replace all the drives, you have a bigger array. I have swapped bigger drives in to fix a cooked drive (why I use big cases now), but never re-sized an entire array.

Thanks for posting your experience. I had built one NAS box to just hold files, and I ended up repurposing it because the power use and noise "worked on my head". Your points about going to sleep and waking go on my big list of RAID problems.

bill
August 2, 2006 4:15:35 PM

I loved this article. It inspired me to retool some old equipment here at my work. I work for the Government so, any tax payers should be happy!

I have an old HP dvd server that is a piece of junk. I have an old dell p4 that is untill just now, worthless and I have a few drives lying here and there. I also have a problem with trying to make an old version of Norton Ghost log into an active directory domain member server.

I see in my mind an ugly yet very useful nas sitting beside my new 2TB San.
August 2, 2006 4:20:01 PM

Quote:
A couple of points you might consider/ criticisms and oversights in your review.

1\. Why not use a motherboard with built in RAID 5 (preferably sata sometimes even with RAID6) and a gigabyte Network port- as this would surely bring down the cost as most modern motherboards have them (by reducing the part count).


Bill: This is a good idea! I've been wondering about swapping in a modern RAID card and this would be a great way to do it. Motherboards with RAID are cheaper than RAID cards.

The reason we did not do this in the current articles is that we wanted to see how much performance we could get from parts already in the closet. I had all the components for this review except 1 300 gig hard drive in my parts cubbord when Tim Higgins suggested it.

Quote:
2\. My main concern with these setups is how much noise they make (especially since many such devices will be used to stream media), so investing in a quiet power supply, cpu fan would seem essential. It would also be usefull to put up benchmarks (subjective or otherwise)as to how quiet the system is against other off the shelf examples.

Bill: Yes! I've been driven to distraction by hard drives enough that I bought 2.5" to 3.5" adapters so that I could run a bunch of old laptop hard drives as my server. But I lost my nerve before building the system.

Also, I don't have a way to measure sound. I'm just a freelancer with a box-o-RAID cards in my closet. And again, we were not writing a scientific article teasing out precise effect sizes. We were pushing to see how we compared against the Teradata.

Quote:
3\. If it is really to be an apples to apples comparrison, why not choose a sata setup to compete with the THECUS NVRAID system (which is better performance wise than the buffallo system).


Bill: I was not loosing sleep over having apples-to-apples comparisons. Information is contained in contrasts. There is so little data accessible to lay-people on RAID5, I was trying to produce the results that I wish I'd seen before I started building RAID systems. How fast can we go for nothing out of pocket? That was the question.

Quote:
4\. The power draw of a system is very important (especially is it going to be an always on system) so I wouldn't want my raid box to be consuming the eqivalent of 5 lightbulbs for occasional file server use. Here again benchmarks of equivalent power draw would be usefull. A smaller power supply would possibly be more appropriate (as an example a mini-itx format Power Supply with power connector splitters for all the drives).


Bill: Good point. If budget had not been a constraint, doing the math (or looking up the draw) on the hard drives would have been step one. Then, buying a fanless power supply would have been step 2.

Quote:
5\. The physical size of the unit is also a consideration (do you really want a hulking great case in the middle of your house). For instance, including other cheap small footprint cases would have been usefull in your review.

6\. To compare it fairly with other off the shelf RAID systems out there, I feel that the ommission of removeable caddies was an oversight (especially against the Thecus products).


Bill: From the cheap and nothing wasted perspective, home caddies being included were an added cost without benefit. As I said, from my perspective running a home business, I'm going to have to drive to CompUSA/BestBuy anyway, a caddy isn't going to REALLY provide me with anything. In an enterprise environment, yes. In a home business environment, no.

Quote:
7\. Another important factor overlooked was boot time/ setup time (how long does the system take to power up and be useable over a network from an off state. Also how long does it take to rebuild an array after a disk failure/ replacement. Benchmarks like these would be very usefull.


Bill: Point taken. I'll talk to the editor about this. We'd need some kind of reference quantity of data to be on the drives for this kind of benchmark to mean anything. The rebuild time on an empty drive will be artificially small. Help me out here, how full should the array be? What kind of data? Would enough video streams to make the drive 70% full be a fair starting condition for the rebuild? The last thing I'd worry about is fragmentation. An array that is fragmented is probably going to take longer to re-build than a freshly minted array. How do we make the fragmentation realistic?

Quote:
8\. Although FreeNAS was dismissed, I feel it was overlooked in 1 critical area- that it can be booted from a USB key. IT is a more efficient system in terms of loaded modules (do youy really want a noveice to work out exactly which packages are necessary from an Ubuntu installation package? -after all this chews up valuable disk space).


Bill: I'm really trying to pull FreeNAS in. Please do not feel that I'm dismissing FreeNAS. On the test hardware it performed slow. On the follow on hardware, I can't even get FreeNAS to install. Let alone install on a USB key. I'm chomping at the bit to INCLUDE FreeNAS, especially if I can put it on a USB key drive! Can you contact me on the back channel and give me some ideas how I can get FreeNAS to install? bill@basicip.com

Quote:
9\. There was also no mention of how easy it is to set up in a headless network state (pretty easy in FreeNAS). There was also no mention of the lack of information/ control panel on the front of the case (that most off the shelf NAS products have),


Bill: I don't think I'm "getting" what you are driving at here. Ubuntu was the desktop version. You've got a full user interface GUI available if you re-plug a monitor/keyboard/mouse to the system. FreeNAS has its menu available unless you enable the root account. I don't have any of the appliance units so I'm not aware what they have available on the front. In my experience when something happens, I need a full console in order to deal with it. So to me, I don't really care. I run eveything headless and then move plugs around.

Quote:
In my opinion, I would not rate this as an effective comparison system. It is probably a reasonable how to article (but litte more) as it fails to address the key areas outlined above. It is the equivalent of showing you how to build a motorised go-cart (with a off the shelf parts) and then comparing it to a car. :o  :o 
[/quote]

Bill: Fair assessment. Except that if Teradata is the car, the go cart is faster and carries more!

Thanks for pushing back man! Iron sharpens iron. I'm feeling the love!

bill
August 2, 2006 4:37:19 PM

This link is pretty straight forward and is an easy read for most explaining RAIDs Nested and not. It sure beats me throwing down books on to the desk and typing it out from the books.

Nesting is not cheap but performance is far greater than RAID 5. IMHO 5 is great for using as much space as possible and still having redundancy, it is best value for your hard earned money.

Wikipedia - RAID
August 2, 2006 5:13:55 PM

All of your systems look expensive to me :D 

I have a PIII 500mhz with 128mb of ram with an Intel 6 Port SATA Raid (bought of Ebay for $86US) controller installed. I run 4 x 320Gb WD hard drives in a RAID 5 config. I did look at some of the other controllers (old 3Ware ones) available on Ebay but decided on paying a bit more for the Intel one as it allows for online volume expansion, a feature which is missing from the cheaper/older cards available and necessary if you ever want to add more storage onto the box without reformating and starting from scratch.

I use an Linux build called NASLite from http://www.serverelements.com/. It was an absolute piece of cake to set up and does everything I could want (except maybe assign specific user rights). Version 2.0 was recently released with RAID support which saved me a lot of money. I have been using version 1 for some time but it did not support RAID and I was getting nervous. I was looking at the Terastation and a few other similar products quite seriously.

Naslite supports SMB/CIFS, NFS, FTP, HTTP and RSYNC and is loaded onto a 64mb memory key. Because my machine is so old I have to boot up off a " kicker disk" first as it does not support booting from USB but in a newer machine you would be able to boot up directly from the USB drive.

It also supports GB transfers but at the moment I only have 100Bt installed as my Router only currently supports 100Base-T. I use the storage primarily for storage and streaming of home video, music and Digital Photo files to my X-Box media centre. It performs this task without any issues. The system is also used with RSync to backup my laptop and desktop hard disk drives. Performance is pretty good and seems to saturate the 100Base-T connection (8-10,000kb/sec). When I go to Gigabit I suspect that I may have to upgrade the system but by that stage I should have an old 2Ghz celeron available for that task.

Installation is a piece of cake (I was a bit nervous of having to go through the whole Linux installation thing). Download the ISO, burn to CD, boot from Cd with USB key plugged in, write OS to key, plug key into system to be used as Nas and boot it to the USB, setting up the kicker disk is also pretty easy.

Initially you will need a monitor connected to set up the network settings etc, thereafter you can see whats going on with the Box by connecting via HTTP, if you want to change any off the settings on the box you need to start a telnet session (accesable from one of the HTTP pages on the server console)

Altogether a very easy to use system.

JohnB
August 2, 2006 5:22:42 PM

Can you do a measure of power consumption? Is it really cheaper if the power draw will even the prices out in a few years?
August 2, 2006 6:34:14 PM

Hey!

I was going to do some simple calculations. But, it appears that the elctricity math is easy to do ... badly.

Here's where I was going to go:

The average PC is said to draw 65 Watts (http://michaelbluejay.com/electricity/cost.html) but looking around at people who are measuring power consumption on systems in-the-same-BALLPARK (http://www.hardforum.com/showthread.php?t=912078) it looks like a better number would be 250 Watts. So a RAID5 PC with a hardware card and 3 extra drives will draw somewhere around 300 Watts. 100 Watts for an hour is 1kWh so a day of 300 Watts of draw is 72 kWh. The average cost in the US per kWH is below 10 cents, so let's say $.10. So the average PC with RAID5 is going to cost something like $7.20 per day to run.

Do we need to consider the 75% efficiency of the power supply? Then this number maybe (help me here EEs!) needs to be divided by .75.

When I try to take components separately this is what I found:

My hard drives draw 9 Watts each (http://www.westerndigital.com/en/products/productspecs....), so per day that is 216 Watt-hours or .216 kWh so each drive in an array costs .216 x $.10 = $.0216. So a 4 drive array costs $.0864 per day to power. Hey, that's not very much (just over 1% of the daily electricity cost), where's all the money going? Of my total 72 kWh to explain, I'm not getting much help from hard drives!

Aha! The Sempron 2200 processor: Approximately 62 Watts or 1.44 kWh per day. Should we factor in the CollerMaster's 75% efficency? If so then we've got 1.984 kWh per day as the wall recepticle sees it. So of my 72 kWh I've got another .... 2 explained.

Given the other components:

RAM:

LSI Logic Megaraid i4:

Power Supply:

Motherboard:

There is no way I'm gong to explain the other 69 kWhs. Hmmm.... time to get a measuring instrument. .... Hope I can trust the readings ...

bill
August 2, 2006 7:00:34 PM

I'm using NASLite v2 on mine with two IDE raid cards, two channels each, and one 120GB drive on each channel. I have all four drives on one Raid 0 stripe.

I don't care about the data failing at this point as I have back ups for anything I want to keep safe. It's is mostly ISO's, videos and MP3s, most of which I have original hard copies stored away.

My main NAS
PII 450
384MB RAM
DLink GbE NIC
2x Promise Technology FastTrak100 TX2 Ultra ATA/100 RAID Card

This machine sits behind my first NAT with port forwarding to the http port so I can get to my stuff from anywhere. I have another DIY NAS machine that keeps personal stuff and it’s behind my 2nd NAT along with my all my other machines.

As for power costs I have five various full size computers running all the time and my electric bill is around $50 a month even with my AC pumping away with all this recent heat. This is in my apartment and I think my costs are quite low on the energy bill. So IMHO the energy cost for and off the shelf NAS might take me down a dollar or two a month, not a big deal breaker to me at all.
August 2, 2006 7:28:14 PM

One thing to consider is that power saving features on a DIY box will cut power usage significantly. Put the HDs to sleep if you’re not hitting the NAS all day. If you’re really anal get lamp timers to turn your equipment on and off 15 min before or after you are to get home or leave and right after you go to sleep, or what ever you do.

I don’t power mine down due to the fact I get my stuff from my port forwarded NAS when I’m at work, over at friends’ family’s places. I don’t know how many times I’ve had to get people a copy of Win XP either because they lost or destroyed the disc; most new PCs do not have a disc with them but only have a key on the box, ect, ect. So I keep a copy wide open via http from my NAS to the web.
August 2, 2006 7:41:34 PM

Ok, I ran a few numbers. Apparently the terastation uses ~56W (per their manual). I have recorded an AMD 64 3200+ system running linux (1 HD) and idle at 120W. So lets assume that at idle (which is where they will spend most of their time) the drives each draw an additional 9W so we now have 152W. Subtract these two and we have a difference of 96W. So, 96W*24hours = 2.3kWh * 365 = 841kWh/yr * .0986$/kWh = $83/year
August 2, 2006 8:24:10 PM

Now how long would a person plan on using a 4 drive RTG or BYOD NAS costing $xxx dollars? A PC can be upgraded far more easily, can be just as quiet, and use just as little power. An of the shelf NAS is dead and useless if the LAN dies on it. How much is a NIC for a PC, hmmmmm.

If you want to save power on the components other than the drives consider a Mini-ITX motherboard with a VIA Eden CPU, some of them have onboard GbE and some other mITX mobos have PCIe slots. If you want quiet they have fanless boards as well.

Mainboards - Mini-ITX (one seller)
August 2, 2006 9:23:17 PM

Billmeade,
Your initial factor in your calculations is wrong, which is why your other calculations aren't adding up.

1000 W for 1 hour is a KWh, not 100 as you used.
So 300w is .3KWh * 24 hours/day = 7.2KSh/day or $0.72/day to run.

You already took the power supply effeciencies into account with your 300W estimate so you don't have to do anything with it in the above calc, but you should use it in the drive and CPU calculations following.

I end up with a quick calculation of about 3.5KWh/day for the components you listed. Considering the 300W draw you started with is only a guess, the 2 are pretty close.
August 2, 2006 9:29:26 PM

Quote:
Billmeade,
Your initial factor in your calculations is wrong, which is why your other calculations aren't adding up.

1000 W for 1 hour is a KWh, not 100 as you used.
So 300w is .3KWh * 24 hours/day = 7.2KSh/day or $0.72/day to run.

You already took the power supply effeciencies into account with your 300W estimate so you don't have to do anything with it in the above calc, but you should use it in the drive and CPU calculations following.

I end up with a quick calculation of about 3.5KWh/day for the components you listed. Considering the 300W draw you started with is only a guess, the 2 are pretty close.


epiecenter has a powersim for calculating your estimated needs. Its pretty neat although it is just an estimate.

http://www.epiacenter.com/powersim/powersim_v2/epiasimu...
August 2, 2006 10:29:43 PM

Quote:
Billmeade,
Your initial factor in your calculations is wrong, which is why your other calculations aren't adding up.

1000 W for 1 hour is a KWh, not 100 as you used.
So 300w is .3KWh * 24 hours/day = 7.2KSh/day or $0.72/day to run.

You already took the power supply effeciencies into account with your 300W estimate so you don't have to do anything with it in the above calc, but you should use it in the drive and CPU calculations following.

I end up with a quick calculation of about 3.5KWh/day for the components you listed. Considering the 300W draw you started with is only a guess, the 2 are pretty close.


PSUs only pull what is being drawn from them. As with most electronics heat is produced, the most heat comes from your components like CPU, Video card, ect. I challenge you to power up an ATX PSU by itself put a paper clip between the green connector to any one of the black connector on the ATX cable set and measure with a volt meter and you’ll find out that a PSU of a certain rating does not pull that much until you put that much of a draw on it.

Think of the PSU by itself as a car in idle, it uses very little. Add CPU and other things it’s like pushing on the gas, it increases the power consumption. Power on demand. :roll:
August 2, 2006 10:33:22 PM

Sub $1500 DIY SFF 1GB DDR, Dual GbE, SATA II, RAID 5, 2.0TB NAS, quiet, and low power. Just add OS.

I found these prices at various locations.

Morex Venus 668B Case - $95.00
Commell LV-673NS Pentium M Mini-ITX Mainboard - $335.00
1GB PC3200 DDR -$56.00
Intel Celeron M CPU 1.30GHZ/1M CACHE/400MHZ FSB SL7RA - $45.00
Hitachi DeskStar 7K500 500GB SATA 3.0Gb/s Serial ATA Hard Drive - 4 x $199.00
HighPoint RocketRAID 2310 PCI Express to SATA II RAID Controller - $145.99
August 2, 2006 10:34:53 PM

Quote:
All of your systems look expensive to me :D 

I have a PIII 500mhz with 128mb of ram with an Intel 6 Port SATA Raid (bought of Ebay for $86US) controller installed. I run 4 x 320Gb WD hard drives in a RAID 5 config.
...........<edit>...........


This looks very interesting. I'm about to retire a similar box (a little higher spec actually), so I could easily add a RAID card and the same as you.
I'm a bit concerned about performance though. Have you measured file read/write performance accross the LAN?

And does anyone with a similar box have a GigE interface to report performance on that? I suspect it wouldn't be great though...
      • 1 / 3
      • 2
      • 3
      • Newest
!