Murphy’s Law tells us that anything that can go wrong will go wrong at the worst possible time. The axiom to this law says Murphy was an optimist. Which begs the question: what happens when RAID fails?
If you're using RAID 5, it means that at least two drives must fail for the array to be broken. If a single drive fails in a RAID 5 configuration, the distributed parity permits the system to continue operating. Some RAID configurations, such as RAID 0 and RAID 1, have no parity drive. As a result, it is more difficult to rebuild the array without all of the drives in working order. RAID 0 stripes data with data blocks on consecutive disks. This is used for faster performance but there is no mirroring and no parity.
With RAID 1 all data blocks are mirrored from one drive to another, If one drive has a physical failure, the second drive can be swapped in to replace it. While there is redundancy, a malware attack on one drive is a malware attack on both. A logical failure to one is a logical failure to both.
RAID 10, sometimes referred to as RAID 1+0, uses striped disks and mirroring, although there is no parity. This approach has the same shortcomings as RAID 1 and RAID 0.
But disks will fail and sometimes, multiple disks fail at the same time. Sometimes this will occur when one disk fails and is not replaced before the second disk dies. At this point, you'll be unable to determine which disk failed first and therefore will have incomplete and outdated data.
Repairing the array will require that both disks be evaluated and possibly both be repaired in order to determine which restores the array’s most recent data, but before that is done, you will likely want to hire a repair depot to conduct the data recovery and repair.
Selecting the right depot can be a daunting task. Unlike finding a qualified technician to repair a failed network infrastructure or damaged database, there are few certifications that specifically address disk drive repair. Instead, says Michael Yasumoto, a senior forensic analyst at Deadbolt Forensics in Beaverton, OR, you must do a thorough evaluation of vendors claiming to have the necessary expertise.
For example, if the disk drive became corrupted or physically damaged due to a cyber attack or possible physical misuse where legal action might be required, the drives being recovered must be done by a technician qualified in forensic recovery and be an expert witness in a court of law, and who can maintain and report on the drives’ chain of custody.
The decision as to whether or not to bring in a forensic specialist, say in an enterprise setting is up to senior management and legal counsel and based on whether potential legal action is possible. Unlike a data recovery task where a file system becomes corrupt and can be repaired with one of the myriad of consumer-class software tools, data damaged due to a deliberate attack that is actionable in court must be recovered through processes that will stand up to a fierce cross-examination by attorneys. Many states, such as Texas, Nevada and Georgia, require that the person conducting the forensic data recovery be licensed as a private investigator (PI). In fact, Texas and Nevada require any data recovery to be done by a licensed PI.
Recovering multiple drives from a failed array usually is more complex than simply repairing a single drive. A typical small to midsize business might have a RAID appliance that will include five drives configured as RAID 5 — four drives acting as the primary storage and the fifth drive serving as the parity drive. Should a single drive fail, the array can be rebuilt using the parity data on the fifth drive. However, if one drive fails and then another fails either before the array is rebuilt or worse, before the IT manager has a chance to rebuild the array, the issue becomes more challenging. The technician recovering the array needs to determine which drive failed first, and therefore is most out of sync with the array.
Scott Moulton is a digital forensics expert who owns Atlanta-based consultancy Forensic Strategies Services and a data recovery company called MyHardDriveDied.com. Moulton, who also trains law enforcement, government agencies and individuals how to do forensic data recovery, says most companies that claim to do data recovery mainly focus on the high-volume, fast turnover recovery that represents 85 percent of the storage recovery market that can be repaired simply using software.
Moulton says repair depots play a numbers game, doing the “easy,” software-focused repairs and turning down repairs that require opening up the drive and replacing damaged parts. Generally opening a drive requires a clean room and perhaps specialized and expensive and more complex tools such as a PC-3000 system for ACE Laboratory in Russia, DeepSpar Data Recovery Systems’ DeepSpar software, or the Atola Insight data recovery tool. Both DeepSpar and Atola Technology are based in the Ukraine.
Although a company that owns these tools is not guaranteed to be able to do data recovery, Moulton says, the fact they know about these tools and made the investment can be a data point in their favor when determining if the depot has sufficient experience in repairing the drive or array. Because it is difficult to compare repair depots, it is essential to ask for and vet references to ensure the depot has explicit RAID expertise, he says.
Before ever engaging a repair depot to recover a failed RAID, the IT manager should check the rest of the disks in the array with any basic tool to ensure there is still data on the drive, he says. Sometimes when one or more drives fail, the array could end up wiping the rest of the array as it tries to recover from the failure. Repairing a failed drive won’t help if the array wipes the data on all of the good disks as well, he notes.
When interviewing potential repair depots, you need to recognize that there is a difference between data recovery and disk repair. If the goal is just data recovery where the hard disk will be trashed after the recovery process, and if the goal is simply to recover the data quickly, it is possible to open drives outside of a clean room environment, Moulton says. While this is not the recommended procedure, sometimes extraordinary measures are needed. Opening a drive outside a clean room will void a warranty and is generally considered contaminated from a forensics perspective, but when successful it can recover data when no clean room is available.
A drive that requires a rebuild due to a failed part or head crash generally will increase the cost of recovery if the repair is done in a clean room but the rebuilt drive still should be trashed after all the data is recovered. With the low cost of disk storage today, it makes more sense to buy a new drive rather than rely on a drive that had already failed once.
You can expect to pay at least $700 for data recovery and perhaps much more if the physical disk must be repaired. A forensic repair, which includes written reports and chain-of-custody security, can further increase the price.
Many commercial programs that claim to do data recovery try to restore the data by writing new data to the original disk. Ideally, experts agree, you will want to create an image of the damaged disk and only work restoring the data from the image on a different, clean drive. That way if important data is accidentally overwritten, you can start over again with a fresh image and the original data is not touched. When the technician is working on the original disk, it should be placed in a system that is write-protected so that the data cannot be destroyed permanently.
Yasumoto says companies that need advanced disk repair should interview companies that claim to have the expertise and ask them to perform a “shadow repair.” By watching how the technician moves his hands over the imagined disk, he says, the potential client who understands the internals of a hard disk can glean some knowledge about how well the technician would do the job on a real drive.
The drawback to this approach, however, is that it takes a lot of time for the vendor who might not want to do the exercise for a prospect who is not paying for the time. Also, for a client that does not understand the intricacies of data recovery, the demonstration would have little value.
Ultimately, repairing a RAID takes individual disk recovery to the next level. Not only must the most recent drive that failed be identified and recovered, but even the good drives in the array must be examined. Because there are no industry standards when it comes to RAID or disk recovery, the selection of the repair depot takes on a new level of importance. Make sure that references are checked and the depot has a data security policy in place.
Editor's Note: In the following chart, we present some essential and basic information about some of the industry leaders in RAID recovery and repair. Several companies did not respond to our request for information. It is also worth noting that in gathering the data for the following matrix, almost every single provider was reluctant to take part, claiming that the recovery and repair industry was full of specious companies, most of whom make false promises. We removed from the charts claims about being the best, or being first.
| Products |
CBL Data Recovery
|
DriveSavers
|
DTI Data Recovery
|
Gillware Data Recovery
|
KrollOntrack
|
MyHardDriveDied
|
Secure Data Recovery
|
WeRecoverData
|
| Pricing |
Go
|
Go
|
Go
|
Go
|
Go
|
Go
|
Go
|
Go
|
| Years In Business | 21 | 28 | 16 | 10 | 29 | 23 | 17 | 11 |
| Number of Technicians | 68 | 26 | 10 | 20 | 125 | 3 | 22 | No data provided |
| Repair Staff Avg Experience | 10 | 6 | 20 | 5 | 10 | 18 | 10 | 17.5 |
| Receiving Locations | 20 locations in 15 countries (not just "receiving or depots") | N/A. 14,000 referral partners worldwide | 3 (plus hundreds of partner locations throughout U.S.) | 1, with 1200 active partners | 19 | 1 | 57: N. America, Australia and UK | 40 in U.S., 10 in Canada, 1 in Mexico |
| Technician Certifications | Multiple certifications | Windows, Apple and Novell certified; PGP, Ultimaco Software, SafeGuard Enterprise, Access Data, Pointsec Check Point, EnCase Guidance, Cellebrite, Blackbag, Paraben | Ace Lab avdanced training, MCSE, Cisco, Apple | No specifics provided | Certified Mac Technician. VMware Certified | CCFS, CCFT, CDRP, DREC, CDRE | Apple Mac Certification, MCPD, A+. SSAE 16 SOC 1 Type II, Class ISO 4 Cleanroom, BBB Accredited with A+ Rating | Mac certified and Microsoft Certtified Professionals. CCE (Certified Computer Examiner), EnCE (Encase Certified Examiner), CHFI (Certified Hacking Forensic Investigator) |
| Average Turnaround Time | 1.5 days | 1-2 business days, plus round the clock priority service | 3 to 5 business days | 6 business days standard, Advantage service 3- 4 business days, Priority service 1 business day | 3-5 business days | 7-10 days | 1-3 days depending on the size and condition of the disk | 5-7 business days, standard service; 2-3 business days, priority service; emergency service is as fast as technically possible; most cases are recovered within 24 hours |
| Turnaround Time Guarantees | Yes | Claim a 97% turn-around time result | Only for Expedite Service | No | No | No | Emergency service options with option for same day service | Yes |
| Vendors Who List Company as Authorized Repair Depot | 2 | 5 SSD manufacturers; 4 notebook/desktop/server manufacturers; 8 storage system manufacturers; 3 hard disk manufacturers | 3, including Western Digital | Dell, Western Digital, Intel for SSDs | Authorized by all major hard drive manufacturers | None | 20+ | N/A |
| RAID Repair Expertise On Staff | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Data Security and Integrity Guarantees | Yes | Yes. Facility undergoes an annual SOC 2 Type II audit, Cisco self-defending network, building monitored with cameras and motion sensors, visitor screening | Yes. SOC 2 Audit | Gillware is SOC 2 Type II security audited. HIPAA compliant data recovery upon request | Yes | Yes | SSAE 16 Type II SOC I Certified with over 40 controls to safeguard customer data physically and electronically | Yes |
| Ability to do Forensic Repairs and Recoveries | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes |
| Technicians with Forensic Recovery Certifications | 7 | 6 | 1 | No data provided | 30 | 2 | 1 | No data provided |
| Technicians With Expert Witness Experience | 3 | 3 | 1 | No data provided | 15 | 1 | No data provided | About 50% |
| Repairs Onsite or Outsourced | Onsite | Onsite | Onsite | Onsite | Onsite | Onsite | Onsite | Onsite |
| RAID Levels Repaired | All | All | All | All | All | All | All | All |
| OS Supported | All | All | All | All | All | All | All | All |
| Average Price for Repair | $1500 | $1600 | $199+ | $700 | $695+ | $800 per disk | Depends | $500-$2500 |
| If You Cannot Do Repair, Is There a Fee? | No | No | No | No | It depends on the system type and service level selected; eg., if priority or emergency services are selected, there is a fee for $65 | Yes, we charge $50 fee per a disk for the evaluation | No | No |