RAID 10 FAIL w/ Intel Rapid Storage Technology

[coldmast]

ASUS P7H57D-V EVO
4x WD Black 1 TB w/ 64 MB cache (windows 7 64-bit is installed on this array)

One drive failed, marked as a smart error.
Replaced bad drive (warranty replaced), array went from degraded (boot-able) to failed (non-boot-able).
The three other drives are still showing up as good (green on SMART).
If anyone is familiar with restoring an on-board/software Intel Matrix Storage Technology RAID 10 array, I would like to know the procedure; since Intel has very little information (their support is a joke) on the subject besides "wipe drives and try again". Redundancy without the ability to restore is not redundancy, so it's my fault for not testing recover-ability of single drive failure before; instead of trusting that a feature would work as advertised.

Anything I have found online recommends to use some RAID recovery software http://www.xtremesystems.org/forums/showthread.php?187723-Intel-ICHxR-RAID-Data-Recovery-101&highlight=Recover%20RAID. Already searched the forums for the big neon sign: couldn't find one; however, links are welcome.

I'm wondering if I'm missing a noob step here?

P.S. criticism welcome; though any advice will pass a logic test before implementing.

 

[coldmast]

I read up on the problem. The array, while in a degraded state, tries to continually rebuild. Apparently there is no way to get it to boot without the failed drive. Something this limiting should be documented by intel. What happens when a drive just up and quits completely, what is the point of having the RAID array in the first place. Recovering the RAID requires a bootable OS running the Rapid Storage Driver. My minimal downtime that I wanted could have been better spent with simple image backups.

 

[Nakecat]

Doesn't sound right, RAID 10 should be able to rebuild even with 2 drivers are down, as long as they are not on the same stripe. And your WD Black 1TB is probably not RE edition. it will fail even it's 100% fine because the lack of TLER.

if you want to run raid with WD driver, make sure you get the RE edition.

 

[coldmast]

Doesn't sound right, RAID 10 should be able to rebuild even with 2 drivers are down, as long as they are not on the same stripe. And your WD Black 1TB is probably not RE edition. it will fail even it's 100% fine because the lack of TLER.

if you want to run raid with WD driver, make sure you get the RE edition.

With the failed drive in it will boot, once a new drive is placed as a standby it will rebuild with the new drive, once the rebuild is started on the new drive the old one can be taken out. The problem is with the rebuild -- there is no option for canceling the rebuild, this could be a firmware related problem. Also, ensure that the latest Intel Rapid Storage Driver is used as I had problems with the motherboard manufacturer's version.

If I was unable to get the failed drive back from my wholesaler, I would have made a boot disk or I would have to set up another drive with windows and the Rapid Storage driver.

Sorry I forgot to simply mention that the drives were WD "Caviar" Black, though if they were RE I would have mentioned that. It is my understanding that error recovery can either be handled by the raid controller or the drive, the Caviar Black ships with TLER disabled (as newer firmware disables this) perhaps because of related troubles.

 

[skytanker2003]

@coldmast: I'm about to do a similar setup. I've got my OS on a separate drive and plan on doing RAID 5 with 3 other drives. Would this problem have been moot if you had had the OS on a separate drive?

I just lost my data once because of Windows Dymanic Disk crapping out when one of my HDDs went bad so I don't want to go to the MB RAID or Intel Rapid Storage if that's just gonna crap out too.

 

[coldmast]

Yes I do believe it is moot. Originally, I also wanted to do a RAID 5 because it maximizes redundancy and storage, but MBR partitions can't go larger than 2 TB; I wish I read up on this: Windows and GPT FAQ.

The problem I had was booting into a degraded array (I needed the 'bad' drive back -- as the array was in a state rebuild), and that I couldn't repair the array from from the SATA/RAID Controller Firmware/BIOS utility. My complaints are that I could not find the documentation (other than the generic "start from scratch"). I also didn't know that I needed to boot to an OS with the Rapid Storage Driver (make sure to download the latest from intel not the motherboard manufacturer's website) to rebuild/repair the array, I thought it would be a simple approach from the SATA/RAID Controller BIOS menu {like in the Rapid Storage menu: plug in drive, mark as spare -- fire and forget}. Don't ever attempt to repair/rebuild from the SATA/RAID Controller BIOS menu, anything I have read stated that it is the surefire way to lose your data.

The RAID Contoller BIOS let me create the array before installing windows which was handy, the Rapid Storage driver should do the same in the Windows OS. Apparently, the driver also lets you mix and match RAID across drives; although, I hardly see the point of this, but to each there own.

If I wasn't able to get the Broken HDD back from the supplier/wholesaler, my next step would have been to create a bootdisk with the Rapid Storage driver. Creating a BartPE/WinPE bookdisk with the Rapid Storage driver would be most helpful in restoring the array. Had that failed, there is always Runtime Software Raid Reconstructor ($) (can also be ran from a BartPE/WinPE bootdisk).