RAID 5 over GIGABIT ETHERNET

[berwicke]

I need to create a video editing storage solution after my move to much larger HD files. I want to do RAID 5 because of its ability to rebuild and the fact that in a four disk solution i can get 3X the speed of a single 7200 rpm drive. right now i have two RAID 0's in my mac pro and am booting off of a esata external enclosure. after looking at everything i have decided i would like to put four 1TB drives in my antec 900 case with a phenom x3, 4gb of ram and a gigabit NIC. my mac pro has gigabit ethernet also.

Antec 900 PC:
500 GB 5400 RPM 2.5" SATA DRIVE with UBUNTU 11.10 (soon 12.04)
RAID 5:
1TB Hitachi 7200 RPM
1TB Seagate 7200 RPM
1TB Seagate 7200 RPM
1TB Western Digital Caviar Black 7200RPM

Mac Pro:
250GB Western Digital SATA boot drive with OS X Snow Leopard 10.6.8
RAID 0:
500GB Seagate
500GB Seagate

my plans are to use the 1tb raid in the mac pro as the fastest data then move projects when im idle with them to the pc raid 5 for speed and redundancy. will i be able to get the potential speed from the drives by limiting myself to gigabit ethernet? also would it be smart for me to edit straight off of the network drive? would it be possible to use both gigabit ethernet ports on my mac pro, and add another gigabit NIC to the pc to double the transfer rate? how would i do load balancing on the mac and ubuntu computers?

my other question, and i know im being needy so thank you to anyone who has taken the time to read down to this point, is how do i setup a RAID 5 in ubuntu as im only a linux novice, and how do i set it up as fully shared network storage.
thanks to anyone who can help.

 

[tomatthe]

WD drives (which are not the RE line) will not work properly in a Raid 5 set unless you bought it prior to ~2010.

You can use bonding on the mac for the 2x 1gbit connections, but you will need a switch which supports this as well.

Raid 5 in ubuntu I can't offer much help on. You should probably do this through your raid controller rather then ubuntu though.

Hopefully thats a bit of help, but you still have quite a few unanswered questions which someone else may help with.

 

[kitsunestarwind]

Has you considered using NAS software like FreeNas or Openfiler
Then you can setup Raid levels on a headless box and access it via shares or iscsi

 

[berwicke]

Has you considered using NAS software like FreeNas or Openfiler
Then you can setup Raid levels on a headless box and access it via shares or iscsi

I had planned on doing something like this, essentially setting it up and then leaving it next to the ethernet switch just powered on. i hadn't heard of Freenas or openfiler before but this sounds good i'll have to take a look.

 

[berwicke]

WD drives (which are not the RE line) will not work properly in a Raid 5 set unless you bought it prior to ~2010.

You can use bonding on the mac for the 2x 1gbit connections, but you will need a switch which supports this as well.

Raid 5 in ubuntu I can't offer much help on. You should probably do this through your raid controller rather then ubuntu though.

Hopefully thats a bit of help, but you still have quite a few unanswered questions which someone else may help with.

what is it that makes the drives not work for RAID 5? i thought the drive was just a slave and had no control over the data it ingested. or is it something with the sata controller? my drive was from early 2010 if i remember correctly.

 

[frombehind]

I have explored these options myself - What I eventually came up with was a NAS (I chose a 5-bay Synology model eventually) and by now I have 2 of them filled with 3tb drives, and everything seems to be going well.

 

[tomatthe]

WD made some changes to the way tler worked on their drives around 2010. Basically you can't change it anymore, and the drives will drop out of raid sets. Google tler western digital raid and lots of information will come up about this.

 

[berwicke]

thanks to everyone i appreciate the input thus far, and glad to know about the wd drive. i usually try to stay away from them as it is, maybe i will just do a three disk raid 5 and keep the wd for a back up of my two RAID 0 500's. i would like to know from those of you who have used a NAS enclosure, what kind of data throughput do you get? i might need to do the double gigabit for t he throughput i need. right now i'm maxing out my RAID 0 setups at only 1/2 my processor capacity (june 2007 8 core 3ghz mac pro). would it be smarter for me to just get two more 500 gbs, do one big four drive RAID 0 and then use a NAS to backup the data on it two two one tb drives?

 

[drumsrule786]

If you really want to go for throughput, get an SSD or two for raid 0 in your mac and load your projects on there while you work on them. If your maxing out a raid 0 setup already don't even think of editing right of a gigabit or even double gigabit network, They will get you around 100 MB/s speed. I don't know too much about ssds and macs so I can't help you there but if you must have absolute speed that would be a good option.

 

[sewalk]

Just how much bandwidth do you expect to get out of Gigabit Ethernet? If you want to get more than about 30-40MB/s, you're going to need to invest in NICs and a switch that can handle the throughput. Jumbo frame support is a must, at least 9k worth. Even with a good gigabit setup, you're not going to see over 80MB/s at best. Even if you are able to bond both ethernet ports on the server you won't get that doubled throughput unless you bond two ports on your client(s). You can run out of switch ports pretty quickly that way.
So if you figure on a maximum of less than 200MB/s over gigabit (160MB/s is really, really good) there's not a lot of reason to focus on pushing RAID performance much higher than this.

 

[CaedenV]

I am getting ready to implement something like this over the summer. Gigabit Ethernet will not be fast enough for HD editing as it is only ~100-120MB/s which is slower than a single internal drive. I mean, it will get the job done... but it is not as fast as I think you would like it to be. Consider internal RAID to get better throughput, but if you are like me and the noise would be an issue then you need to look at a few other options:
1) a long KVM wires to keep the computer in a different room
2) bonding or ganging 2 gigabit ports for a total of ~200-240MB/s which is acceptable for HD editing
3) look for businesses that are upgrading old 4Gb/s fiber that they would be willing to sell you. This would give you ~400MB/s and would be more than fast enough for a good long time (this is what I am attempting to do). The only down side to this is that you will have pro grade equipment which will be more complicated to work on and maintain.
4) use a pair of internal SSDs in RAID-0 (~900MB/s of throughput) and do local editing, but depending on your space requirements this could get expensive very quickly... and you would need to back it all up on another device in the event of something going terribly wrong.

 

[sewalk]

Gigabit Ethernet will not be fast enough for HD editing as it is only ~100-120MB/s

Hardly. You can't just divide the signalling rate of 1,000,000,000 by 8 and declare that 1000TX will deliver 125MB/s. Ethernet simply isn't that efficient. Using a switch instead of a hub will at least eliminate CSMA/CD overhead, which can get very high depending on the number of nodes in the collision domain. The problem remains, though, that ethernet simply isn't that effecient. Every frame sent has a preamble and delimiter, a header containing source and destination addresses, the data payload, followed by error detection code. Then once we look at the payload, we still have overhead for the network protocol being used (IP, most often) which has more overhead of its own. Then we have another protocol layer, like TCP or UDP which has some amount of overhead of its own and still yet another layer, in this case, for SMB (aka Windows networking and file sharing). With all these layers, efficiency starts to drop off. 75-80% is about as good as it gets. Under the best possible conditions, with nothing but top quality hardware and cabling and no extraneous network traffic, 100MB/s is the most you will ever see. Add a couple of other nodes to the network and any traffic at all and you'll get real-world performance of no better than 90MB/s with 80MB/s being more typical. Bond two ports together and you'll see some more overhead, maybe another 5% or so. Now you're at 76MB/s per channel or 152MB/s.
Ethernet isn't all that pretty when you lift up the hood and look at the inner workings.

 

[FireWire2]

I need to create a video editing storage solution after my move to much larger HD files. I want to do RAID 5 because of its ability to rebuild and the fact that in a four disk solution i can get 3X the speed of a single 7200 rpm drive. right now i have two RAID 0's in my mac pro and am booting off of a esata external enclosure. after looking at everything i have decided i would like to put four 1TB drives in my antec 900 case with a phenom x3, 4gb of ram and a gigabit NIC. my mac pro has gigabit ethernet also.
....

Certainly, all things are possible!

To do what you need, you must have a hardware raid either PCI E controller or SPM393, SPM394; this will reduce the IRQ sending to CPU via RAID engine as motherboard RAID and allows your system run lots more efficient.

Add TWO more Intel NICs and bonding them connect direct to you MAC w/o Gb switches.

You can get about 125- 160MB/sec depend on your set up and usage protocol.

 

[berwicke]

thanks for all the support everyone! i think i've decided that what i'm going to do is a 3 drive RAID 0 with three of the one tb drives (two hitachis and one samsung) and then use the one tb WD and the two 500gb seagates for backup of the more important files inside the pc. i figure if i do all my editing locally like it was suggested and i had originally planned on, then i can just move my older finished projects at a decent speed over the ethernet to the file server.