Unusual RAID5 write activity

[_locutus_]

I recently built a software raid5 array using an Adaptec 1430sa PCI-e controller and 4 Seagate ES2 1TB hard drives. I decided to use Windows Server 2k3 to manage the array rather than the controller itself to prevent data loss if the controller dies and I can't find an identical replacement.

Here's the problem. When I transfer a single file (700MB) to the array from the network I get a write speed of around 110MB/sec but after the transfer is complete the hard drives continue to be accessed for some time, presumably due to parity data being calculated. If I transfer a series of files the first file writes fast, 110MB/sec, but then the following files transfer very slowly, around 5-10MB/sec for the remainder of the transfer.

I read several other forum entries asking about similar speed issues. One user suggested building the array on the software controller instead of in Windows, and that this would significantly increase performance.

My question is this, if I did rebuild the array again on the controller or in the controllers software, would this really improve performance? I just don't want to rebuild the array on the controller for another 20 hours to find that there is no difference then spend another 20 hours going back to a Windows software solution.

Also, what block size would you recommend for ideal performance with large sized files, 350MB+

Thanks for reading, I am eager to hear your suggestions.

Tyler

 

[enlightenment]

You're using a host-based controller in RAID5 using Windows, that's asking for trouble IMO. If you want RAID5 on Windows, you need to have a good controller, such as Areca ARC-1210.

If you're on a budget, the Intel ICHxR RAID5 driver is pretty okay too. And if you have the fortune to be able to use Linux and FreeBSD in particular, you can use Software RAID5 with write speeds in excess of 400MB/s (starting with 8 drives).

Your controller is mainly a SATA controller with bootstrap support and a Windows driver which does the actual RAID. It may use your RAM als write-back buffer, so that writing the first x MB's are fast, but after that it has to fallback on the RAID5 array's actual write throughput, which can be as low as 4MB/s on crappy RAID5 implementations. Since this will also cause your disks to seek excessively, they will also be using much more power, generate more noise and potentially have a shorter lifespan.

Those host-based controllers are good for RAID0, RAID1, RAID0+1, just not for parity RAID. Not because it's the parity that has to be calculated, but because of the complicated 2-phase write design weakness that RAID5 has. Any intelligent RAID5 driver will have to 'save up' write request and combine them into chunks of a very specific size.

So i would say your options are limited to:
- migrate to RAID0+1 with 4 disks (you loose 1 disk more to redundancy)
- buy a new controller
- migrate to a linux/FreeBSD platform

 

[_locutus_]

Thanks for the quick response, I see what you are saying. I was under the mistaken impression that in a software raid setup the cpu would handle all the calculations that the hardware raid card would normally handle, but obviously thats not the case.

The reviews for the Areca ARC-1210 look impressive, I may give that one a shot. One other question, if I go with the 1210 and decided to expand the array at a later date can I add a second card and create a 6 or 8 disk array? Or would I have to replace the card with something like the ARC-1220?

 

[enlightenment]

Hm, to clear some things up:

Software RAID
Provided by Operating System or Driver, connected to SATA controller with RAID-support disabled if applicable). Offers great flexibility but no boot support; with the exception of RAID1 which behaves like a normal disk. -> your CPU does everything

Fake RAID / Driver RAID / Onboard RAID / Firmware RAID / Hybrid RAID (yeah those things are ALL the same)
SATA controller advertized as "RAID controller" which is actually a normal SATA controller with a bootstrap addition, allowing booting from a driver-based RAID device. Once booted, the (often Windows-only) RAID driver is loaded which does all the work -> your CPU does everything

Hardware Assisted Controller
Much like a true hardware RAID controller, but 'off-loads' the complicated RAID5 combine/split/XOR operations to your systems CPU, thus saving the need for a powerful processor on the RAID controller; making it a cheaper alternative. An example of such a controller is the quite-popular Highpoint RocketRAID 23xx series, which have a nice pricetag, the 2340 offers 16 ports and they all offer reasonable RAID5 support and quite a bunch of features. They also have drivers for Linux and FreeBSD. -> your CPU does the heavy work but only in RAID5; otherwise the same as a True Hardware RAID Controller

True Hardware RAID Controller
A real controller where an onboard processor on the RAID controller does all the work. Many of such controllers are currently equiped with an Intel IOP series processor, which offers RAID5 and RAID6 support and especially the newest ones offer great performance. -> Your CPU does no extra work than it would have to do for a normal disk; all RAID-related processing is done on the RAID Controller's processor

The Areca ARC-1210 and 1220 are somewhat dated already, since they feature a 500MHz Intel IOP332/333 processor, while newer models like Areca ARC-1231ML which have an 800MHz Intel IOP341 processor. the 1231ML is a 12-port controller and using Mini-SAS connectors instead of normal SATA (hence the ML:Multi Lane). This saves space since one big cable will feed 4 SATA-disks. The Highpoint RR2340 also has these Mini-SAS->SATA connectors.

If you pick an Areca controller, you get a real hardware controller with solid RAID5 support, onboard buffercache and excellent features and performance. You can easily expand by adding disks to your RAID5 array. Just make sure your controller has enough ports.

One other thing, pay attention to the spin-up current harddrives use when powering up. Each disk uses up to 35W, so you may find yourself having a system that powers off not long after you pressed the On-button. With a good 400W Power Supply Unit (PSU) you should be able to feed 8 disks, but you might need some more if you're going to feed twelve. And always pick a 80-Plus certified PSU.

That's it, good luck

 

[_locutus_]

Thank you for the extensive explanations.

Due to the expense of the hardware solutions I've decided to go with something with 8-ports for expandability to a larger raid5 or raid6 in the future. I was looking around newegg for different Areca controllers and found an ARC-1222. At only $50 more cost than the 1220 it has the newer 800MHz Intel processor and DDR2 533 memory as opposed to the DDR333 memory on the 1220. Here is the link on newegg:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816151039

My question i about the ports, 2x SFF-8087 ports. Are those the Mini-SAS ports you described in your earlier post? If so this card should come with adapter cables to attach standard SATA drives/cables correct?

Any other input on this card would be appreciated.

 

[enlightenment]

Yeah each of those ports can feed 4 SATA disks, so with 2 of those you get a controller suitable to connect 8 SAS or SATA disks. You can connect SATA to a SAS controller, but not the other way around (SAS drive on SATA controller won't work).

I don't know for sure, but i trust the package will contain the cables you need. Ask your supplier to be sure.

Some info on the ARC-1222 can be found here:
http://www.webconnexxion.com/raid/product_info.php?cPath=86&products_id=723&osCsid=eeb379084fb57fe5f723eb13623114aa

The owner of this Dutch webshop is also kind enough to give you any advice, even if you don't buy there.

I warned you against too high power drain when harddrives are spinning up, but the Areca controllers have a feature known as Staggered Spin-up, and gracefully spinning the drives up in sequence instead of all at once, thus removing the need for special concerns about your power supply. Just pick an efficient one, Amacrox 400W Fanless works great in my NAS and is much more efficient than other PSU's (89%).