Best Way To Carve Up Storage and Serve It Up On Windows 2008 R2 Server

Hello,

I'm working on provisioning 2 x Promise Vtrak 24-Drive arrays. One is a master with dual controllers and the JBOD expansion is attached via SAS cable. The array will be directly attached to a Windows 2008 R2 Server via 4 x Fiber ports. I have carved two 12-drive RAID6 LUNS on each chassis for a total of 4 available LUNS.

My question is that I noticed that the windows server sees all 4 logical disks on the Promise Arrays as separate disks about 18TB each. Should I create each disk as a separate simple disks or use spanning on all 4 LUNS to make one large volume for simplicity even though have 4 different drives for storage would be the end of the world.

The idea of one large volume via spanning or striping seems really cool but the whole layering of a software span/raid over the 4 x RAID6 LUNS really scares me, software raid over a hardware raid. Is this a recommended practice and is it reliable? I'm thinking there could be a performance hit by doing this. I've search for a several hours on the internet in various forums and articles and didn't really find a conclusive answer since most talked about setting it up on disk that was built into the physical server itself. What's the worst that could happen if the server went belly up? Would my data be intact if I had to rebuild the server?

Thanks for any help. Seems like the more I read and learn about storage the more confusing it's becoming because there are so many ways to setup storage.

Ok since your using enterprise class equipment then don't bother with the build-in windows dynamic disk / volume striping. Carve out your LUNs based on your site requirements and then map them to the appropriate disk / folder inside the server. What you have built seems like a miniature SAN without the FC switch. I'm running a very similar setup but with Solaris and SUN equipment.

Does your SAN storage processor support LUN expansion? What you might think about doing is to create a smaller RAID5 LUN and assign several 2~3 disks into a "hot spare pool". We've found this method more space efficient then to have double parity data for each array. Overtime you can expand the size of the LUN based on requirements and the storage processor should redistribute slices and parity info transparently.

The thing with LUN's is once you have data on them its difficult to reduce / reassign them. Generally speaking each LUN represents a single storage device to the host OS. NTFS allows for dynamic expansion but not reduction of volume, some third party tools allow you to reduce a NTFS volume.

RAID6 is overkill in your situation, from what I understand you have 2 x 24 disk arrays? That is 48 disks total. Are you using each 24 disk array as a single LUN? Have you allocated for a hot-spare-pool? HSP is very important as their a group of standby disks that just stay there waiting. Should one of your RAID members have a fault or become unavailable the one of the disks from the HSP is used as a stand-by to replace the failed drive. This makes RAID-6 unnecessary as RAID6 is just RAID5 with double parity bits.

The way we did things here is similar. I have a SE3510 array with 12 146GB 15K FCAL disks. I created a single RAID-5 array of 10 disks with 2 HSP's and mapped it to a single LUN. The array is connected to the Sun system through four fiber lines to two HBA's for 8Gbps bandwidth. Using mpxio in Solaris to handle the multi-pathing and link aggregation. The SE3510 has two storage processors and each one controls two channels and presents its own LUN ID to the host OS. Both processors control the same drives but their able to work together to distribute the I/O.

Does your array have a storage processor inside it? Is it multiple (A/B) and what mode does it operate in (active/passive or active/active)? That effects your performance and bandwidth utilization. How much disk space does your intended application need or is there multiple applications? Different applications have different storage characteristics and a single volume might not be the best idea to store everything.

Is the server just being used as a generic file server, is it doing virtualization, is there a need for a database store, exchange information store, ect..

The reason there is so many different ways is there are different requirements. Like if you needed a generic file server, exchange information store, SQL server backend for a website. You could do virtual machines to host each application and give them different luns. A large 10 member RAID5 LUN for the file server, smaller 5~6 member RAID5 LUN for the SQL server, 4 ~ 5 member RAID5 LUN for the exchange server and just a RAID-1 LUN for the website. Allocate 2 disks per array inside the HSP and your set.

Normally arrays of that size are used in a SAN environment where you have 4~5 different physical machines accessing the data over a high speed 4~10gbps SAN. Its looking like you only want all 48 disks on a single server?

Just a small thing to clear up about the Hotspares because there not being used on a regular basis the plates arn't spinning and the arm isn't moving no data is being written or read more or less. So yes there is some wear and tear on them but its really very minimum

Your mostly correct except RAID 5/6 don't protect against read failures but against a disk going south. Bit errors will happen but the logic inside the storage processor will detect it and correct it on the fly, adding additional parity bits won't solve that. The only security layer that RAID6 provides over RAID5 is that two disks can simultaneously die (or be pulled out) and your data is still accessible and safeguarded. It has to do with disk failure rates not bit error rates (BER).

Raid 5/6 does nothing for your BER, that is what the storage controller's error checking and correction is for. The reason to use RAID6 is if your doing very very large arrays where single disk redundancy doesn't cut it. Perfect example is your setup with that 24 member array. At 22 disks (assuming 2 HSP) the chance of a single disk going bad is pretty high, the chance at two is small but sill within the realm of possibility (especially if their all from the same production lot). You would definitely do RAID6 on a 22 member array but not on a 10 or smaller array.

Since your use is as a single large file store this becomes really simple. Can the first chassis (master) detect and control the disks from the second chassis? I know our Sun StorEdge's can do that, the storage processors can control the disks off the expansion module.

The reason I ask is that what your looking for is RAID 60. On our Enterprise SAN (not the SUN one) we're using a DELL EMC solution that is a dual storage processor with 12 disks and has four expansions modules attached each with 24 disks. The storage processor manages all the disks and can slice / export LUN's to however we want. What we did was create a bunch of smaller 5-member RAID5 luns and then bundle them together in a RAID50 (spanning) to different servers based on usage need. This allowed us to grow the array by tacking on another RAID5 LUN into the RAID50 array. This gets very complicated and I don't think you need something quite this complex.

You should make a 22 member RAID6 array for each chassis, then couple them together in a RAID 0 array. Its basically a RAID0 composed of members in a RAID6 configuration. Your storage processor should support this. It will create a single "super LUN" that the host OS will see as a single storage volume. It will be high performance with high availability. Do yourself a favor and specific two HSP disks per chassis. These are vital for when (not if) a disk goes bad, the HSP disks will stand in while you replace the faulty disk. You can actually have two disks go bad at once and both HSP's will stand in and your array will keep on trucking like nothing happened. You replace both disks and the array will rebuild the two replacement disks and return the stand-by disks to the HSP.

The alternative is to have multiple drive letters which you mentioned wanting to avoid. Although you can avoid that problem using DFS root and name spaces, but if your on an "Apple" enterprise network ... god have mercy on you.

Ok a quick summary of disk RAID with respect to HSP.

RAID0 -> 1 Disk failure and your data is gone
RAID1 -> 2 Disk failures and your data is gone
RAID3/4/5 -> 2 Disk failures and your data is gone
RAID6 -> 3 Disk failures and your data is gone
RAIDz X -> X+1 Disk failures and your data is gone. Solaris with ZFS use's RAIDz mode where you specify the number of parity bits you want and it calculates out the rest.

On a RAID5 array with 10 members, you have a member disk experience a failure. There is parity information available and the storage processor is able to synthesis the data that would be missing. The host OS doesn't even know a disk is bad and all data read requests are returned with valid data, but slightly slower. The disk has a blinking light next to it to inform the administrator that there is a dead member and to replace ASAP.

Now here is where a HSP enters in. A RAID5/6 running in a degraded state not only takes a hit in performance but also is vulnerable to data loss if another disk dies. This degraded status will remain until the administrator replaces the failed disk and the storage processor rebuilds the array. The role of a HSP is to provide a replacement disk the moment the disk failure takes place, the rebuild process happens automatically the moment one member fails (or is removed). This is known as "standing in". The rebuild happens in the background and your data is back to being protected as soon as possible. This gives the administrator time to order a replacement disk from the vendor. The administrator places the new disk into the same spot the failed disk resided and the array will detect it and automatically start "fail-back" where the data from the HSP is cloned over to the new member disk and the array is rebuilt. After rebuilding the stand-by disk is returned to the HSP to be used again.

Its a very VERY good idea to have a HSP with 1~2 disks regardless of the type of array you have. HSP's protect your data ~after~ a disk failure, they prevent the nightmare scenarios of multiple disks failing in sequence due to them being from the same production lot. They've personally saved my bacon after we had a disk failure. One disk failed then the next day another one in the same array failed. If we didn't have a stand-by disk then we would of had system downtime and had to resort to older disk backups on tape.

Thanks man, your help tonight has been invaluable RAID60 spanning is what I'm looking for then. I've run through the manual a bit and it didn't go into great details. I will try to talk to one of their support staff tomorrow. I'll let you know how it turns out.

Hat's off to you man!

I looked at the Manual and it didn't go into much detail in regards to RAID 60, but he specs say: RAID 60 in addition to Automatic Array Rebuild, Array Expansion, Array Migration, Bad Sector Remapping, Dedicated Hot Spares, Hot Plug, Hot Swap.

There's no data on the arrays so I have the luxury of "playing" with it without worrying about destroying data. I will test out RAID 60 tomorrow.

Depends on the quality of the arrays and the method that their being linked. From the way you described it the second unit was acting as a drive expansion to the first, in this case they can be considered a single unit. There shouldn't be any logical reason not to treat them that way.

Try to map out the possible situations where a chassis goes down. Power being pulled, bad PSU (does it have redundant PSU's), the cable being disconnected. What kind of warranty does this have on it and how long is it for? If you split your data up and a chassis goes down you still lose access to data just not all of it. And if the main chassis goes down you lose access to everything anyway as that contains the storage processor. So really we're only discussing what happens if the expansion chassis goes offline.

Honestly while you can protect against single or even multiple disk failures, protecting against a physical enclosure is nearly impossible. The only solution is to have 4~5 enclosures and use RAID amongst them, which gets ridiculously expensive and complicated.

Also it sounds like they didn't do a good job of explaining things to you. RAID0 striping if one member goes down you lose the data, but RAID0 spanning if a member goes down you only lose the data stored on that member. Otherwise run a disk repair utility and the volume will come back online. I think they immediately assumed you were talking about RAID60 striping, or they don't support the spanning version of that implementation.

I called promise and their systems do not support RAID0 Spanning only striping. I thinking I'll just go with two 22 drive raid6 with 2 dedicated spares in each chassis.

That's exactly what I did. I mount the shares on the server and the two volumes are transparent to the end user. In the end it makes no difference technically weather there is more than on volume on the server.

Thanks for your continued help sir