RAID 5 vs RAID 3 for Video files

jfurterer

Distinguished
Nov 21, 2008
80
0
18,640
I do a fair amount of video editing and encoding and I've finally gotten around to looking into an alternate solution to my 3x external hard drives. I have an old full tower that supports 6 internal drive bays and plan to use an old 80GB drive in on of them as an OS disk.

There seems to be allot of support for RAID 5 however I'm sure if this is the right solution for large files.

The way I understand RAID 5 to work is that it uses a computation to produce parity across all the drives allowing one drive to fail and the integrity of the data to still be intact. The one big advantage I see is that you don't loose out on allot of disk space. However I've read that it's inefficient with large file sizes and that should a drive happen to fail rebuilding the replacement drive can be an all day affair.

I've caught snippets of RAID 3 and heard this suggested as a superior RAID solution for large files. The way I understand this to work is that instead of doing a computation to create the parity one dedicated disk is used.

My questions about the two are these.

Do they both support varying drive sizes?
How much space is "lost" do to parity with each solution?
What is the performance gain with a RAID 3 over a RAID 5 when storing large (200mb and up) files?

Anybody who has had experience with RAID 5 or RAID 3 or both would be appreciated. Also I plan to use a control card(s) to handle the raid as my mobo only supports 0, 1 any suggestions on this front would be welcome as well.
 

ShadowFlash

Distinguished
Feb 28, 2009
166
0
18,690
Well first of all, this link should sum up the reason I would never recommend RAID 5....

http://miracleas.com/BAARF/RAID5_versus_RAID10.txt

It's pretty heavy reading I know, but well worth it....

Here is an old link for RAID 3....it's not very popular any more...

http://searchstorage.techtarget.com/tip/0,289483,sid5_gci1077519,00.html

RAID 3 sounds perfect for your usage, but the inherent flaws make it very important you use it only for this purpose, and you must be very regimented keeping it defragmented. I can't say I've ever used RAID 3, so this is all "in theory". If you have the room for the drives ( and can afford the capacity overhead ), RAID 10 provides a very nice general purpose array.

No RAID level plays nicely with varying capacity drives. The larger capacity drives will default to the smallest capacity drive size. Also, the overall array performance will generally be limited to your slowest drive. With RAID 3 however, your fastest drive ( if mismatched ) should always be your parity drive, as this is your limiting factor. If you go with 3, it might be cost/performance effective to purchase a very fast drive of similar capacity just for your parity drive. Both RAID 3 and 5 lose the capacity of 1 drive, no matter how many drives are used. RAID 10 loses the capacity of half your drives. This is why some use RAID 5, as capacity is greatly improved.

Now, on to the questions....

You have an old external case ? What kind of interface to the computer (SCSI, USB, Firewire, E-SATA, Fibre Channel )? What kind of drive interface ( SCSI, ATA, SATA, Fibre Channel ) ? If you are using SCSI, then what speed, interface and cabling ?
And, most importantlly what is your budget ?

 

jfurterer

Distinguished
Nov 21, 2008
80
0
18,640
I haven't read your articles yet but plan to today. However, to answer your questions.

The machine would likely be networked. I have an old Athlon 64 that I'd use and the mobo has 4 sata connections. I therefore would be spending on a control card. We have a small linux controlled render farm here and I talked the art director into authorizing set up of a central storage hub for completed renders. Completed renders off the farm would be sent to these new machine.

Currently I have 2 old 500gb drives and a 320gb drive. Since you indicated that RAID doesn't play nice with different drive sizes I'd likely purchase 4x1tb drives to start and put the 500s in another machine as a RAID 1.

My budget is fairly flexible since it's on the company. However if I spend more they'll want to see more as in capacity. Performance isn't as important as capacity and redundancy. The farm is unusually qued up over night and when we get back in the morning we review final renders. We have a few RAID 1 set ups but a larger RAID 1 wasn't acceptable as we'd loose to much capacity. One of my colleges had mentioned RAID 5 and that's when I started down this path.
 

ShadowFlash

Distinguished
Feb 28, 2009
166
0
18,690
Well, what you are looking at is setting up a NAS ( netorked attached storage ). While this will definately work, the performance of the array will almost definately limited by your network speed, not the RAID level. Look into the real world transfer rates of large files through 10/100/1000, and you probablly will be dissapointed. Using dual gigabit cards and jumbo frames can help. As far as RAID levels, RAID 3 offers the same capacity of RAID 5. A good RAID 3 controller card will probablly cost you a pretty penny though ( think $500-$700 minimum, $1000 for a really nice areca ) as this level is now only supported on high level cards AFAIK. If you feal like gambling with RAID 5, it is very cost effective....both in controller cost and capacity ( just read the link I posted to understand the risk ). Both would more than likely out-perform network speed with a 4 drive set-up. It's unfortunate that the capacity trade-off for RAID 10 is too great for you. What is the typical file size being transfered ? I can tell you that once the numbers get really big ( 20+ GB ) network transfer can be obnoxiously slow. I'm not a linux guy, but I know windows can have severe problems transfering file sizes larger than available RAM. There is a program called TeraCopy that helps alleviate it. I frequently move 150+ GB directories full of 1-4 GB files across a dual gigabit network with jumbo frames, and I can tell you, its not fun.