Murphy’s Law tells us that anything that can go wrong will go wrong at the worst possible time. The axiom to this law says Murphy was an optimist. Which begs the question: what happens when RAID fails?
If you're using RAID 5, it means that at least two drives must fail for the array to be broken. If a single drive fails in a RAID 5 configuration, the distributed parity permits the system to continue operating. Some RAID configurations, such as RAID 0 and RAID 1, have no parity drive. As a result, it is more difficult to rebuild the array without all of the drives in working order. RAID 0 stripes data with data blocks on consecutive disks. This is used for faster performance but there is no mirroring and no parity.
With RAID 1 all data blocks are mirrored from one drive to another, If one drive has a physical failure, the second drive can be swapped in to replace it. While there is redundancy, a malware attack on one drive is a malware attack on both. A logical failure to one is a logical failure to both.
RAID 10, sometimes referred to as RAID 1+0, uses striped disks and mirroring, although there is no parity. This approach has the same shortcomings as RAID 1 and RAID 0.
But disks will fail and sometimes, multiple disks fail at the same time. Sometimes this will occur when one disk fails and is not replaced before the second disk dies. At this point, you'll be unable to determine which disk failed first and therefore will have incomplete and outdated data.
Repairing the array will require that both disks be evaluated and possibly both be repaired in order to determine which restores the array’s most recent data, but before that is done, you will likely want to hire a repair depot to conduct the data recovery and repair.
Selecting the right depot can be a daunting task. Unlike finding a qualified technician to repair a failed network infrastructure or damaged database, there are few certifications that specifically address disk drive repair. Instead, says Michael Yasumoto, a senior forensic analyst at Deadbolt Forensics in Beaverton, OR, you must do a thorough evaluation of vendors claiming to have the necessary expertise.
For example, if the disk drive became corrupted or physically damaged due to a cyber attack or possible physical misuse where legal action might be required, the drives being recovered must be done by a technician qualified in forensic recovery and be an expert witness in a court of law, and who can maintain and report on the drives’ chain of custody.
The decision as to whether or not to bring in a forensic specialist, say in an enterprise setting is up to senior management and legal counsel and based on whether potential legal action is possible. Unlike a data recovery task where a file system becomes corrupt and can be repaired with one of the myriad of consumer-class software tools, data damaged due to a deliberate attack that is actionable in court must be recovered through processes that will stand up to a fierce cross-examination by attorneys. Many states, such as Texas, Nevada and Georgia, require that the person conducting the forensic data recovery be licensed as a private investigator (PI). In fact, Texas and Nevada require any data recovery to be done by a licensed PI.
Recovering multiple drives from a failed array usually is more complex than simply repairing a single drive. A typical small to midsize business might have a RAID appliance that will include five drives configured as RAID 5 — four drives acting as the primary storage and the fifth drive serving as the parity drive. Should a single drive fail, the array can be rebuilt using the parity data on the fifth drive. However, if one drive fails and then another fails either before the array is rebuilt or worse, before the IT manager has a chance to rebuild the array, the issue becomes more challenging. The technician recovering the array needs to determine which drive failed first, and therefore is most out of sync with the array.
Scott Moulton is a digital forensics expert who owns Atlanta-based consultancy Forensic Strategies Services and a data recovery company called MyHardDriveDied.com. Moulton, who also trains law enforcement, government agencies and individuals how to do forensic data recovery, says most companies that claim to do data recovery mainly focus on the high-volume, fast turnover recovery that represents 85 percent of the storage recovery market that can be repaired simply using software.
Moulton says repair depots play a numbers game, doing the “easy,” software-focused repairs and turning down repairs that require opening up the drive and replacing damaged parts. Generally opening a drive requires a clean room and perhaps specialized and expensive and more complex tools such as a PC-3000 system for ACE Laboratory in Russia, DeepSpar Data Recovery Systems’ DeepSpar software, or the Atola Insight data recovery tool. Both DeepSpar and Atola Technology are based in the Ukraine.
Although a company that owns these tools is not guaranteed to be able to do data recovery, Moulton says, the fact they know about these tools and made the investment can be a data point in their favor when determining if the depot has sufficient experience in repairing the drive or array. Because it is difficult to compare repair depots, it is essential to ask for and vet references to ensure the depot has explicit RAID expertise, he says.
Before ever engaging a repair depot to recover a failed RAID, the IT manager should check the rest of the disks in the array with any basic tool to ensure there is still data on the drive, he says. Sometimes when one or more drives fail, the array could end up wiping the rest of the array as it tries to recover from the failure. Repairing a failed drive won’t help if the array wipes the data on all of the good disks as well, he notes.