Sign in with
Sign up | Sign in
Your question
Closed

RAID 1 - Mirroring three drives?

Tags:
  • NAS / RAID
  • Backup
  • Servers
  • Storage
Last response: in Storage
Share
June 27, 2007 5:28:19 PM

Is it possible to set up a RAID 1 system so that it mirrors onto three drives? I've never run or setup a RAID 1 system before so I'm hoping someone knows from experience.

The idea behind this is I'm part of a small company that wants to upgrade to a RAID server/backup system. Ideally, we would have a server with three identical drives all mirrored. One would be taken offsite for disaster prevention purposes and brought into the office periodically to be updated. The other two would sit in the server at all times so that they would backup each other incase one goes down and we don't lose a week's worth or work for example. With this system, we wouldn't be at risk of losing data incase a HD fails and we would also have a relatively updated offsite backup as well. Can this be done? From what I know about RAID, this would be the only system that would allow offsite backup as well as mirrored in-office HDs.

More about : raid mirroring drives

June 27, 2007 5:38:50 PM

You can only use two drives with RAID 1.
a b G Storage
June 27, 2007 5:51:32 PM

As belvdr says, RAID-1 is for a pair of drives. You could set up a RAID-1 with two drives, and back them up to a third drive, which could be an external drive. If you get two external drives, you can do a full backup weekly to one of them (which is then kept off-site), and a "differential" backup to the other one every day. A differential backup copies anything new or changed since the last full backup, but doesn't reset the Archive attribute, so each day you'll get all files that were added or changed since the weekly backup was done. This reduces the number of backups that must be searched to two, the last full and the last differential; there's no need to wonder when a file was created or changed. The differential backup drive can also be a lot smaller, since it will not include files that never change, like application executables and DLLs.
Related resources
June 27, 2007 6:08:50 PM

you cant mirror 3 drives, however to do what you want do a 2 drive raid 1 then get a 3rd drive as an external usb drive and use a backup program to backup data to that drive.
June 27, 2007 7:12:16 PM

Agree, I'd RAID 1 two of the drives and then have the 3rd one be an image of the RAID, which would be imaged every X days.
June 27, 2007 7:54:37 PM

RAID 1 configuration can be done with 2 HDDs.
If your server hard disks are hot swap what you can do just have a 3rd hard drive the same capacity as the RAID1 and while is working just swap one of the hard drives with the 3rd one. It will copy everything while you are working and in case and when is finished you will have your OS and data on your spare hard drive. In case something happens to both drives you can still use the 3rd one to start your server.
June 27, 2007 8:24:51 PM

depending on how smart or support the RAID HW is you can some times add Spare disks (RAID 1 + an spare hdd) so if an disk fails it auto start to rebuild with the spare disk (norm its used in RAID 5 the spare disk but i think it work on RAID one as well just if the software/hardware supports it)
June 27, 2007 8:38:32 PM

You can mirror as many as you like, if your controller supports it. I have 4 in raid1 on my NAS's. These are the boot partitions for the OS. So the machine will boot even if 3 drives fail.
June 28, 2007 2:19:03 AM

Quote:
RAID 1 configuration can be done with 2 HDDs.
If your server hard disks are hot swap what you can do just have a 3rd hard drive the same capacity as the RAID1 and while is working just swap one of the hard drives with the 3rd one. It will copy everything while you are working and in case and when is finished you will have your OS and data on your spare hard drive. In case something happens to both drives you can still use the 3rd one to start your server.



Thats not a good way to do it, it will work, but its stressful on the drives to be rebuilt like that and a good way to corrupt data.


@leegx he wanted to be able to take a drive offsite for backup, so a hotspare wont help with that.


@blue a 4 drive mirror isnt a raid 1, its a raid 10 or a raid )+1 depending on whether its stripped then mirrored or mirrored then stripped.
June 28, 2007 1:15:30 PM

Quote:
a 4 drive mirror isn't a raid 1, its a raid 10 or a raid )+1 depending on whether its stripped then mirrored or mirrored then stripped.


My drives are broken up in to 2 partitions. The first partition on each drive is in RAID1 (mirror, boot section). the second part is in RAID 5 (data redudency) so I do not have a 10. This configuration is widely used in NAS's. They use a watchdog bios. If one drives fail to boot with in 5 min, it moves to the next.

The point is depending on the controller and/or software you can have a mix.

The question was could he have 3 raid 1. The answer is yes.
June 29, 2007 3:27:54 AM

thats still not a 3 drive raid 1, thats a raid 1 and a raid 5, unless your saying you have multiple drives mirroring the first, but really you still shouldnt be putting more then one array on a drive, cause then you degrade multiple arrays with a single failed drive. Most controllers do not support more then 2 drives in a mirror set.
June 29, 2007 4:40:50 AM

I think what your'e talking about is RAID 0+1, which can use odd #s of drives.
June 29, 2007 12:17:53 PM

Its possiable to make a 8 drive RAID 1 if you want, provided your controller supports it. But what a waste of space. It is not recommended to be swapping drives out of a raid 1 for backup. Thats adds undue stress on the whole system.
June 29, 2007 1:10:53 PM

There is no limit to the number of drives you can mirror, we commonly run with 3 drive mirrors, the question is do the costs involved justify the expense of doing it.

If your looking for high performance the controller you use needs to be able to make concurrent writes to every disc in the set, so for three disccs the controller needs to be able to make three concurrent writes, same for reading, the controller would need to be able to make concurrent reads for every disc in the set. If the controller you use cannot do this then you will get no peformance increase over smaller mirror set.

It can provide a very high level of redundancy, in that if one disc in the mirror set fails it allows you to maintain a mirror while you swap out the failed disc, although if redundancy is what you want you may be better of looking at a RAID 5 array for a solution. You would not be losing as much disc space as with a three disc mirror set.

Maybe you should look at a NAS/SAN system that can replicate to a remote NAS, most vendors provide some replication capability. All your servers could access the drives (which could be in an array) and the data could be automatically mirrored to the remote array.
June 29, 2007 10:11:00 PM

Thanks for all the good replies. Good to know that it's at least possible. In response to jstall, yeah, we're looking at getting a NAS. Performance is not an issue with us and the costs are trivial considering you can find 500 GB SATA-II drives for around $100 these days. As mentioned in my first post, our ultimate goal is to marry the redundancy of a RAID-1 system with the ability to keep a weekly or bi-weekly updated mirror offsite. I know with many NAS' that you can attach a USB drive to it and back up to that but the scenario I was envisioning was being able to bring the "off-site" drive into the office, pop it into the NAS which already has two drives set up for RAID-1 in it, and having the off-site drive be updated automatically to then be taken off-site again. So for that short time, there would have to be a RAID-1 array with three drives in it although most of the time it would only have to support 2 drives. So like I said...I know there's the external USB option but to me it would seem that the sheer simplicity of popping in a drive, waiting a couple hours, and then pulling it out is unmatched. Some have mentioned that the stress of doing something like that isn't a good idea but what good is a RAID if simply cloning/updating a drive compromises the whole system??

BTW, the NAS we're looking to get is the Infrant ReadyNAS NV+. From what I can tell it's a feature-rich NAS and supports everything we need. Feel free to comment on it.
June 29, 2007 11:17:09 PM

According to my manual, (readynas600) you'd have two ways to do this... 1 way would be to set up a two drive mirror with a hot spare... In your case it would be a cold spare most of the time, but once configured the readynas firmware is pretty tolerant... Bring your off-site drive in, plug it into the hot spare slot, wait for it to initialize, and pull one of your active drives... Patiently wait for the mirror to rebuild, and put your off-line active drive back, wait for the mirror to rebuild again (should be real quick...) and your mirror should now be slots 1 & 2 active, slot 3 hot standby... Pull your standby and take it back offsite. A bit tedious, but...

Another method would be to have two drives as a mirror, and a third drive just sitting there, so to speak. When installed and recognized, do a manual back-up (over write) from your mirror to your third drive.

Go to the infrant website and download a soft copy of your manual, while there ask this as a support question. Their support is excellent, registered user or not. You'll probably have a suggestion from them in your email that day. (There is a phone option I believe, but I'm in Aus....)

As to the product itself, I'd love to be able to justify getting the nv+ myself... If I could find a buyer for my chassis, I would. You'll not be disappointed
June 29, 2007 11:52:27 PM

NAS is not connected via USB its connected over the network. What you could do though is have a system with a mirror and then use norton ghost or acronis true image to image the drive and save it to a NAS unit that you can take offsite.


@timck, did you read that review, it said that drobo wasnt suited for business use, drobo is $500 with no drives which is expensive.

@croc, pulling a drive and rebuilding to the other spare, its really stressful on the drive because it rebuilds everything, the chance of this drive failing is a lot higher, which isnt good for a backup drive.
June 29, 2007 11:54:32 PM

Quote:
Its possiable to make a 8 drive RAID 1 if you want, provided your controller supports it. But what a waste of space. It is not recommended to be swapping drives out of a raid 1 for backup. Thats adds undue stress on the whole system.


Is that a specialized chip similar to the Intel Matrix or something? Not even my HP EVA's or the EMC's have the ability to do that (or at least not that I have seen).
June 30, 2007 12:03:07 AM

The HP MSA units can, but as previously stated why would you?
June 30, 2007 12:09:54 AM

Quote:
Its possiable to make a 8 drive RAID 1 if you want, provided your controller supports it. But what a waste of space. It is not recommended to be swapping drives out of a raid 1 for backup. Thats adds undue stress on the whole system.


Is that a specialized chip similar to the Intel Matrix or something? Not even my HP EVA's or the EMC's have the ability to do that (or at least not that I have seen).


I thought the same thing until i did research as no Dell controller, or lsi logic or adaptec controller supports that.
June 30, 2007 12:37:31 AM

Quote:
The HP MSA units can, but as previously stated why would you?


Well, I guess you could say the EVA's could do that too, now that I think about it. The EVA's have virtual arrays, not really an array across particular physical disks, and so your RAID 1 would span multiple drives.

I agree, it seems a waste to me.
June 30, 2007 12:38:16 AM

I'm not recommending either method, and personally I'd have to think that transporting a drive back and forth itself would increase the risk of failure. I run my NAS in raid 5, and if the house burns down I've lost the lot.

I have had several USB external drives, their lifetimes aren't usually very long either...

But the OP seems to think that an off-site drive is a good idea, and at $100 for a .5TB drive, well its affordable.

Were I to attempt this I'd use the 2nd method that I mentioned. The infrant platforms are really flexible, in that until you assign a drive to be something else, (ie, part of an array) then its in JBOD mode.

I also personally wouldn't try to make a NAS a boot device.
June 30, 2007 12:49:43 AM

Well using NAS for offsite is probably the best way because you can plug that into any network and be up, with a hard drive alone, you need a computer to plug it into. But i dont trust hard drives as backup for critical stuff anyway...I will be using on my new server raid 5, then imaged to NAS then use ntbackup system state to tape.
June 30, 2007 1:33:43 AM

Quote:
I also personally wouldn't try to make a NAS a boot device.


Wouldn't that be a tricky endeavour, not to mention a terribly slow boot disk. :) 
June 30, 2007 1:58:24 AM

Not practical for a SOHO or SMB.... We do boot off of arrays at work, but they have dual fibre channel cards in the servers, and dual dual-channel controllers in the arrays... Expensive, but fast and reliable.

Most SOHO / SMB NAS solutions have a single channel ethernet port, thus not practical for speed / reliability reasons.
June 30, 2007 2:14:07 AM

Um, I meant that as a more humorous question, not reality... ;) 

I never boot off arrays. I always use local disks for OS and the arrays for app and data storage. It's a habit I guess, as I see no additional benefit in converting.
June 30, 2007 5:36:08 AM

28 sec. start to finish, loading hpux, oracle 9i, initialising all the tables and taking new data is not really all that slow... :wink:
June 30, 2007 6:03:07 AM

you running gigabit? on mine im thinking about using freenas and using dual nics and aggregate links to go 2 gigabit.
June 30, 2007 6:27:32 AM

I prefer mirroring the OS on two small drives (the OS doesn't change that much) and then either mirror or RAID 5 separate data drives. This way you can image with Ghost or similar utility to an image file on a bootable DVD or image to a spare off-site drive. Then just backup your data to a USB attached drive(s) as often as you like. This will allow for a single drive to fail and still be up and running while also allowing for your disaster recovery with the off-site drives.

Also, on the data backup to the USB drives you may want to do tandem backups for taking off-site and leaving one at the office for quick recovery of corrupt or user deleted files without having to go get the backup drive from the other location. This makes the users very happy, instant restores. :wink:
June 30, 2007 6:27:52 AM

At home that wouldn't be a bad way to go, just make sure that the nics are capable of 'teaming' so that if you lose one the other takes the primary role. I still wouldn't use that as a boot drive, personally, as there are still too many SPOF's for a system that I have to depend on for 24 x 7 support.

The systems that I was referring to are typically HP 'N' class servers, with something like the HP MSA 1000/1500 drive array attached. The MSA units boot rather nicely over fibre-channel.
June 30, 2007 5:10:10 PM

yeah i wont be booting from the NAS its just to backup, i am in the process of building a video server the motherboard is a tyan dual socket 771 with intel nics so they can be teamed and my switch is a Dell powerconnect 2708 web managed switch that does pretty much everything except jumbo frames. Though its probably excessive for a home file server its more fun they using consumer level stuff.
July 5, 2007 2:07:07 AM

are looking for is called a tape drive. Carrying HDDs around and such is all very well, but not really the answer for backp purposes.

In addition as mentioned, plugging drives in and out of RAID arrays is still inherently risky and will seriously impact performance when you are doing a RAID rebuild (note it won't just update changes to the disk, it has to recopy block by block every bit state on the source drive)
July 29, 2007 7:11:02 PM


Heres the whole cake. I think I got the setup you trying to do.

Make a partition on the 3rd disk with the same size as in the mdraid.

run command

# mdadm /dev/md0 -a /dev/sdc3
where md0 is the raiddevice and sdc3 is a satadisk partition 3.

If you run the command

# mdadm --detail /dev/md0

You see something like this

--------------------------------------------
/dev/md0:
Version : 00.90.03
Creation Time : Tue Dec 26 19:16:41 2006
Raid Level : raid1
Array Size : 20972736 (20.00 GiB 21.48 GB)
Device Size : 20972736 (20.00 GiB 21.48 GB)
Raid Devices : 2
Total Devices : 3
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Sun Jul 29 20:53:04 2007
State : clean
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1

UUID : 367f461d:63371a04:8655586c:a2ec96bb
Events : 0.224

Number Major Minor RaidDevice State
0 3 3 0 active sync /dev/hda3
2 3 67 - active sync /dev/hdb3
1 8 35 1 spare /dev/sdc3

------------------------------

You see, that in case of faulty hda3 or hdb3, sdc3 will take their place.

Now simulate a faulty drive (either hda3 or hdb3 in my case) with command

# mdadm --manage --set-faulty /dev/md0 /dev/hdb3

What happens is that hdb3 sets to faulty drive and sdc3 will now be synct up with hda3.

---------------------
Number Major Minor RaidDevice State
0 3 3 0 active sync /dev/hda3
1 8 35 1 active sync /dev/sdc3

2 3 67 - faulty spare /dev/hdb3
----------------------

check progress with
# cat /proc/mdstat

When it synct up, just put it back

Remove the device
# mdadm /dev/md0 -r /dev/hdb3
Then add it again
# mdadm /dev/md0 -a /dev/hdb3

Now you have hdb3 as spare, if you want it back as it was, set the one you want to remove to faulty or just remove it with mdadm -r.

piece of cake :) 
July 29, 2007 7:23:06 PM

offcourse you make a script that do this from cron a certain time every day or why not make a webpage-button @ the intranet "Sync backup" :)  just ensure that the usb-disk will get the same devicename everytime...
July 29, 2007 8:45:43 PM

The same company that makes this also makes a 3.5" version:
http://www.tomshardware.com/2007/07/26/storage_accessor... nd_pros/page2.html

There are also competing products.

You should be able to take two drives that are on a RAID 1 rack, and add them to the system as a single drive. Add another drive the same size as a single drive to the system. Use the system's RAID controller to do a RAID 1 between the 2-drive rack and the third drive. Make the third drive removable via a standard single-drive removable bay.
September 13, 2011 3:37:33 PM

To the original question, yes, here's how. You can choose between 2 disks in RAID1 with a hot spare that will only sync if something goes wrong with the original pair. Or you can have 3 disks in RAID1 (or more). Anybody who says otherwise is wrong, at least on a recent Linux kernel.


Start perhaps with a normal 2-disk RAID 1:

# cat /proc/mdstat
md1 : active raid1 sdb3[1] sda3[0]
87889848 blocks super 1.2 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk


Duplicate partition table to new disk (in my case, sdd)

# sfdisk -d /dev/sda > /tmp/partinfo
# sfdisk /dev/sdd < /tmp/partinfo

Add sdd as a hot spare:

# mdadm --add /dev/md1 /dev/sdd3

Or, remove it as a hot spare, GROW THE RAID 1 to 3 DISKS, and add the new physical disk.

# mdadm --remove /dev/md1 /dev/sdd3
# mdadm --grow /dev/md1 --raid-devices=3
# mdadm --add /dev/md1 /dev/sdd3

After the resync,...

# cat /proc/mdstat

md1 : active raid1 sdd3[2] sdb3[1] sda3[0]
87889848 blocks super 1.2 [3/3] [UUU]
bitmap: 1/1 pages [4KB], 65536KB chunk


# mdadm --detail /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Mon Sep 5 23:53:00 2011
Raid Level : raid1
Array Size : 87889848 (83.82 GiB 90.00 GB)
Used Dev Size : 87889848 (83.82 GiB 90.00 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent

Intent Bitmap : Internal

Update Time : Tue Sep 13 23:31:29 2011
State : active
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0

Name : quad:1 (local to host quad)
UUID : 49249722:ca58f4eb:82802f4c:4bdc4954
Events : 1323

Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3
1 8 19 1 active sync /dev/sdb3
2 8 51 2 active sync /dev/sdd3
February 5, 2012 12:09:46 AM

This topic has been closed by Mousemonkey
!