Raid Primary disk out of sync

JoeHoughton

Reputable
Sep 15, 2014
6
0
4,510
Hi,

Our server was not responding to anything this morning (Network, USB, anything).
So had to hold the power button down. On restart, there was an endless loop of Windows Startup recovery. We couldn't get in to windows, due to this loop, even selecting boot window normally and other ways.
In the end got a message saying "System Volume Corrupt" from Startup recovery.
To get around this (and to get the company working again), we took one of the harddrives (HDD0) and plugged it straight into the MOBO and avoided the Raid SAS controller in the PCI slot.

We looked at it at lunch time and it says that the Hard drive we have is now out of sync, but it says HDD1 is Primary. I am a bit worried that syncing will wipe all the work people have done since this morning.

I need to run chkdsk on the HDD0, as I have a load of logs saying so. Anything else that anyone can suggest?

Thanks in Advance,

Joe

Windows Server 2008 R2
PERC 6/i Integrated SAS Raid Controller card from Dell.
 
Solution
Yea dell servers BIOS haven't changed that much over the years on looks.

Probably got a PERC H200 or 310 or something along those lines. So at this point gotta be careful. Usually what I do is take the drive that is out dated on data and remove it form the RAID and plug in the one with the lastest files. Make sure you can boot up off that. If you can great.

Then I would take the other drive, and delete all partitions leaving it with out any partitions on it and then plug it back into the SAS Controller and assign it to the RAID 1.

A few other thing you can do 1) Download LSI Mega Raid Software and install it on the server along with Dell OpenManage if you don't have it already installed. I always make sure I have both of those...

JoeHoughton

Reputable
Sep 15, 2014
6
0
4,510
Hi drtweak,

Sorry for the late reply. It is a Dell Poweredge T310. I believe it is Raid 1 although it was all set up before we received the server and I can not see it written anywhere in the Raid card BIOS.

The BIOS is frankly appalling.
 
Yea dell servers BIOS haven't changed that much over the years on looks.

Probably got a PERC H200 or 310 or something along those lines. So at this point gotta be careful. Usually what I do is take the drive that is out dated on data and remove it form the RAID and plug in the one with the lastest files. Make sure you can boot up off that. If you can great.

Then I would take the other drive, and delete all partitions leaving it with out any partitions on it and then plug it back into the SAS Controller and assign it to the RAID 1.

A few other thing you can do 1) Download LSI Mega Raid Software and install it on the server along with Dell OpenManage if you don't have it already installed. I always make sure I have both of those installed on all our Dell servers since the Dell PERC Cards are Branded LSI Cards. You can use OpenManage or MegaRaid to assign the Hard drive to the RAID (This is a bit easier to deal with that the RAID BIOS

or 2) Use the RAID Bios.

Now if you do it though software you can just boot off the updated drive only, then after its up and running plug in the other drive. It should see that you're running off the current drive and then sync that drive to the added one. doing it this way you shouldn't have to do anything if everything is setup correctly. You can use the two software programs to check to see if its rebuilding or if you need to manually start the rebuild.


Now first and foremost....DO NOT DO ANYTHING UNTIL YOU DO A FULL BACKUP OF THE ENTIRE SYSTEM! If you have Windows Backup up and running great. Do a Backup before you do this and make sure its a 100% FULL BACKUP! This way if everything is lost for what ever reason you have a full backup to recovery the ENTIRE system from. If you don't use that make sure you clone the drive, and then boot off that cloned drive to make sure everything is there. Don't blame me if stuff blows up lol

 
Solution

JoeHoughton

Reputable
Sep 15, 2014
6
0
4,510
Many Thanks drtweak. I started the backup but came across disk errors so had to run CHKDSK last night which was a slow process. Will follow the rest of your instructions tonight.
 
Yea a corrupt file system will stop ya there lol did you do a chkdsk /r or a /f? Don't need to do a /r unless yo know you have bad sectors.

Also since they are plugged into the motherboard for the time being trying installing crystal disk info. It can read the smart status of the hard drives but it can't read though the newer dell PERC Cards. Also are they SATA or SAS Drive you have in your server?
 

JoeHoughton

Reputable
Sep 15, 2014
6
0
4,510
Needed to do a /r. Managed to get non-corrupt data off it for our backup. Pretty sure the drive is failing as we are still finding corrupt files and need to redo the chkdsk.
Tried booting off the other Hard drive now we had a full non-corrupt backup - this drive is worst than the first. Couldn't even boot off it.

This means both hard drives on our RAID 1 system are failing / have failed.
On our good(I should say best hard drive) CrystalDiskInfo gives us warnings of "05 - Reallocated Sectors Count" and "C5 - Current Pending Sector Count"
Looks like we need two new hard drives for this. Can't believe both have failed at the same time!

Edit - They are Sata hard drives but the Controller card is a PERC 6/i SAS. I presume the cables we have got convert the two?
 
Yea get those drives replaced asap! Take the least back of the two, toss it back in the RAID card, add a new drive to it and add it to the RAID 1. Hopefully it can rebuild to the new drive! As for backups when you do you do? Do you use a 3rd party program or windows backup and if windows backup is it a FULL windows backup? Like OS and everything? If so i would just restore from there. Otherwise get a new drive and add it to the better of the two drives in the RAID 1. Hopefully it can resync before the other drive goes bad. Then once its done remove old drive, replace with new, and add to the RAID 1. You can use the Mega Raid Utility for all of this VS using the RAID BIOS. Easier to deal with.

And then once that is all done BUY A 3RD DRIVE AND ASSIGN IT AS A HOT SPARE! That way when a drive starts to fail like this again (and if you usually won't know until you reboot the server or if you have Dell Open Manage Installed) if a drive is failing or if the drives are out of sync. If you have a hot spare then it will just kick that drive in when another drive fails.

At this point you got to be careful. Make sure you have backups of everything because if that one good drive goes and you don't have a full windows backup then you might be SOL and have to reinstall Server and restore files.

And yes SATA will work on SAS but SAS Will NOT work on SATA.