Sign in with
Sign up | Sign in
Your question

Best Way To Carve Up Storage and Serve It Up On Windows 2008 R2 Server

Last response: in Storage
Share
June 7, 2011 3:07:52 AM

Hello,

I'm working on provisioning 2 x Promise Vtrak 24-Drive arrays. One is a master with dual controllers and the JBOD expansion is attached via SAS cable. The array will be directly attached to a Windows 2008 R2 Server via 4 x Fiber ports. I have carved two 12-drive RAID6 LUNS on each chassis for a total of 4 available LUNS.

My question is that I noticed that the windows server sees all 4 logical disks on the Promise Arrays as separate disks about 18TB each. Should I create each disk as a separate simple disks or use spanning on all 4 LUNS to make one large volume for simplicity even though have 4 different drives for storage would be the end of the world.

The idea of one large volume via spanning or striping seems really cool but the whole layering of a software span/raid over the 4 x RAID6 LUNS really scares me, software raid over a hardware raid. Is this a recommended practice and is it reliable? I'm thinking there could be a performance hit by doing this. I've search for a several hours on the internet in various forums and articles and didn't really find a conclusive answer since most talked about setting it up on disk that was built into the physical server itself. What's the worst that could happen if the server went belly up? Would my data be intact if I had to rebuild the server?

Thanks for any help. Seems like the more I read and learn about storage the more confusing it's becoming because there are so many ways to setup storage.

June 7, 2011 3:46:22 AM

Ok since your using enterprise class equipment then don't bother with the build-in windows dynamic disk / volume striping. Carve out your LUNs based on your site requirements and then map them to the appropriate disk / folder inside the server. What you have built seems like a miniature SAN without the FC switch. I'm running a very similar setup but with Solaris and SUN equipment.

Does your SAN storage processor support LUN expansion? What you might think about doing is to create a smaller RAID5 LUN and assign several 2~3 disks into a "hot spare pool". We've found this method more space efficient then to have double parity data for each array. Overtime you can expand the size of the LUN based on requirements and the storage processor should redistribute slices and parity info transparently.
m
0
l
June 7, 2011 4:19:38 AM

@palladin9479 Thanks for the quick reply. I'm not quite certain if the Promise systems supports dynamic LUN expansion, I will check. We just received it last week so I haven't had too much time with it. I thought about make each Array 24-Disk Raid6 too, so I would have two big LUNS to work with, what are you thought on this? Also, what about make the two Arrays one big LUN? Don't know if it's a good idea.

I agree with you, I'm not too keen on using the Windows built-in dynamic disk options, seems more like something that would be useful for a stand alone server with no raid. I recall doing it years back but it was with just two drives.

I will probably call Promise to tomorrow too see what they recommend. Feel free to respond again please.

Thanks again!
m
0
l
Related resources
June 7, 2011 5:19:18 AM

The thing with LUN's is once you have data on them its difficult to reduce / reassign them. Generally speaking each LUN represents a single storage device to the host OS. NTFS allows for dynamic expansion but not reduction of volume, some third party tools allow you to reduce a NTFS volume.

RAID6 is overkill in your situation, from what I understand you have 2 x 24 disk arrays? That is 48 disks total. Are you using each 24 disk array as a single LUN? Have you allocated for a hot-spare-pool? HSP is very important as their a group of standby disks that just stay there waiting. Should one of your RAID members have a fault or become unavailable the one of the disks from the HSP is used as a stand-by to replace the failed drive. This makes RAID-6 unnecessary as RAID6 is just RAID5 with double parity bits.

The way we did things here is similar. I have a SE3510 array with 12 146GB 15K FCAL disks. I created a single RAID-5 array of 10 disks with 2 HSP's and mapped it to a single LUN. The array is connected to the Sun system through four fiber lines to two HBA's for 8Gbps bandwidth. Using mpxio in Solaris to handle the multi-pathing and link aggregation. The SE3510 has two storage processors and each one controls two channels and presents its own LUN ID to the host OS. Both processors control the same drives but their able to work together to distribute the I/O.

Does your array have a storage processor inside it? Is it multiple (A/B) and what mode does it operate in (active/passive or active/active)? That effects your performance and bandwidth utilization. How much disk space does your intended application need or is there multiple applications? Different applications have different storage characteristics and a single volume might not be the best idea to store everything.

Is the server just being used as a generic file server, is it doing virtualization, is there a need for a database store, exchange information store, ect..

The reason there is so many different ways is there are different requirements. Like if you needed a generic file server, exchange information store, SQL server backend for a website. You could do virtual machines to host each application and give them different luns. A large 10 member RAID5 LUN for the file server, smaller 5~6 member RAID5 LUN for the SQL server, 4 ~ 5 member RAID5 LUN for the exchange server and just a RAID-1 LUN for the website. Allocate 2 disks per array inside the HSP and your set.

Normally arrays of that size are used in a SAN environment where you have 4~5 different physical machines accessing the data over a high speed 4~10gbps SAN. Its looking like you only want all 48 disks on a single server?
m
0
l
June 7, 2011 6:16:11 AM

@palladin9479

The reason why I'm leaning towards raid6 is because the change of read failure increases as drives size increases, in my case 2TB drives. You seem well versed in RAID's but if you search raid5 vs raid6 you'll see may pages talking about the dangers of running raid5 and how the single parity has bitten many people on rebuilds because of drive error and they end up losing all their data. I know raid is not backup but I'm looking for the really good protection at the cost of a little IO.

Our storage has Intel Xeon C5500/C3500 processors. The controllers are active/active and yes I've setup multpathing so the controllers can handle the load balancing. I have one 4 port Qlogic HBA in the server (8Gbps) the controllers have 4 ports each, I'm running two ports from the HBA to one controller and the the other two to the other controller.

We're using the storage strictly as a fileserver and nothing else. We will be copying 30+ TB's to it as-soon-as we decide how we're going to carve it up. I know it's not good practice but the customer who will be using this storage will not have another copy of the data on any other server, maybe some USB backups of various stuff. It's not my call it's just what the customer does. I've been tasks to provide a lot of storage and I know they're going to run this thing until it dies so I want to provide a high level of redundancy, which is why I chose raid6.

Speed shouldn't be an issue since only about 10 clients will be access the file server, maybe only a handful of the 10 at a time. The data will consist mostly of millions of jpeg files. We do data collection which involves tons of jpeg frames to capture the data, the files average several MB's each.

Yes we have 2 x 24 arrays, here is a link to the models http://promise.com/storage/raid_series.aspx?region=en-U...

I'm possible thinking about using each 24 disk array as a single LUN so the there will be only be volumes to share out. I guess I'm thinking it might be cumbersome to deal with 4 drive letters if I serve the LUNS up with the 4 12 disk LUNS that I have now. We're primarily a Apple shop so Apple's XSAN is different in that it has dedicated metada controllers and more than one server can write to the volume at the same time without any issue. I didn't set ours up but we have one big 72TB volume and all 4 servers write to it at the same time.

My thoughts regarding the hotspare is RAID5 with hot spare is essentially RAID6 with only single parity. RAID6 would use more drives specifically depending on how many LUNS you have. The thing about RAID5 hotspare is the hotspare drive is always running so in a sense it's wearing down, so if it kick in say a few years later it's a drive that's been running for several years even though no data has been written to it, no?

Again, I'm primarily looking to create the most amount of space with good protection assigned to only one server.

Thanks for being patient with my Q & A's. I'm not new to storage but still learning a lot everyday.
m
0
l
June 7, 2011 6:40:10 AM

Just a small thing to clear up about the Hotspares because there not being used on a regular basis the plates arn't spinning and the arm isn't moving no data is being written or read more or less. So yes there is some wear and tear on them but its really very minimum
m
0
l
June 7, 2011 6:42:12 AM

@illfindu, thanks for the clarification regarding the hotspares.
m
0
l
June 7, 2011 6:43:33 AM

Your mostly correct except RAID 5/6 don't protect against read failures but against a disk going south. Bit errors will happen but the logic inside the storage processor will detect it and correct it on the fly, adding additional parity bits won't solve that. The only security layer that RAID6 provides over RAID5 is that two disks can simultaneously die (or be pulled out) and your data is still accessible and safeguarded. It has to do with disk failure rates not bit error rates (BER).

Raid 5/6 does nothing for your BER, that is what the storage controller's error checking and correction is for. The reason to use RAID6 is if your doing very very large arrays where single disk redundancy doesn't cut it. Perfect example is your setup with that 24 member array. At 22 disks (assuming 2 HSP) the chance of a single disk going bad is pretty high, the chance at two is small but sill within the realm of possibility (especially if their all from the same production lot). You would definitely do RAID6 on a 22 member array but not on a 10 or smaller array.

Since your use is as a single large file store this becomes really simple. Can the first chassis (master) detect and control the disks from the second chassis? I know our Sun StorEdge's can do that, the storage processors can control the disks off the expansion module.

The reason I ask is that what your looking for is RAID 60. On our Enterprise SAN (not the SUN one) we're using a DELL EMC solution that is a dual storage processor with 12 disks and has four expansions modules attached each with 24 disks. The storage processor manages all the disks and can slice / export LUN's to however we want. What we did was create a bunch of smaller 5-member RAID5 luns and then bundle them together in a RAID50 (spanning) to different servers based on usage need. This allowed us to grow the array by tacking on another RAID5 LUN into the RAID50 array. This gets very complicated and I don't think you need something quite this complex.

You should make a 22 member RAID6 array for each chassis, then couple them together in a RAID 0 array. Its basically a RAID0 composed of members in a RAID6 configuration. Your storage processor should support this. It will create a single "super LUN" that the host OS will see as a single storage volume. It will be high performance with high availability. Do yourself a favor and specific two HSP disks per chassis. These are vital for when (not if) a disk goes bad, the HSP disks will stand in while you replace the faulty disk. You can actually have two disks go bad at once and both HSP's will stand in and your array will keep on trucking like nothing happened. You replace both disks and the array will rebuild the two replacement disks and return the stand-by disks to the HSP.

The alternative is to have multiple drive letters which you mentioned wanting to avoid. Although you can avoid that problem using DFS root and name spaces, but if your on an "Apple" enterprise network ... god have mercy on you.
m
0
l
June 7, 2011 7:01:08 AM

Our new storage controller supports raid60, but will this setup allow for expansion of the volume? the Promise array supports up to 6 expansion chassis. The reason I was entertaining using the spanning in Windows Server in the first place was the ability to easily expand the volume. But from what you wrote you seem to be do the spanning on the storage controller correct? Yes I know, there's nothing enterprise about Apple's enterprise network products;-)

Here are some specs from the Promise raid sorry for the crappy formatting, doesn't work too well for posting.

Raid Level Support 0, 1, 1E, 5, 6, 10, 50, 60
64K, 128K, 256K, 512K and 1MB
Global or dedicated with revertible option
1024
32
• Advanced Cache Mirroring over PCIe Gen2 • Simple, drag-and-drop LUN Masking and Mapping • Asymmetric LUN Unit Access (ALUA) • Volume Copy
• Media Patrol • Background Synchronization • Foreground Initialization • Rebuild • Redundancy Check
• Predictive Data Migration (PDM) • Intelligent Bad Sector Remapping • SMART Error Handling • NVRAM Error Logging
• PerfectFlash - Non-Disruptive Software Update • I/O performance & power monitoring tools • USB Service Log • LDAP Support for central user management
• Disk SMART Polling • Online Capacity Expansion (OCE) • RAID Level Migration (RLM) • UPS Monitoring • Feature rich task scheduler for background activities
• Disk Slot Power Control • Read/Write Check Table • Write Hole Table
• Four levels of advanced power management disk drive (MAID) support • Efficient 80Plus Certified power supplies
m
0
l
June 7, 2011 7:01:50 AM

Ok a quick summary of disk RAID with respect to HSP.

RAID0 -> 1 Disk failure and your data is gone
RAID1 -> 2 Disk failures and your data is gone
RAID3/4/5 -> 2 Disk failures and your data is gone
RAID6 -> 3 Disk failures and your data is gone
RAIDz X -> X+1 Disk failures and your data is gone. Solaris with ZFS use's RAIDz mode where you specify the number of parity bits you want and it calculates out the rest.

On a RAID5 array with 10 members, you have a member disk experience a failure. There is parity information available and the storage processor is able to synthesis the data that would be missing. The host OS doesn't even know a disk is bad and all data read requests are returned with valid data, but slightly slower. The disk has a blinking light next to it to inform the administrator that there is a dead member and to replace ASAP.

Now here is where a HSP enters in. A RAID5/6 running in a degraded state not only takes a hit in performance but also is vulnerable to data loss if another disk dies. This degraded status will remain until the administrator replaces the failed disk and the storage processor rebuilds the array. The role of a HSP is to provide a replacement disk the moment the disk failure takes place, the rebuild process happens automatically the moment one member fails (or is removed). This is known as "standing in". The rebuild happens in the background and your data is back to being protected as soon as possible. This gives the administrator time to order a replacement disk from the vendor. The administrator places the new disk into the same spot the failed disk resided and the array will detect it and automatically start "fail-back" where the data from the HSP is cloned over to the new member disk and the array is rebuilt. After rebuilding the stand-by disk is returned to the HSP to be used again.

Its a very VERY good idea to have a HSP with 1~2 disks regardless of the type of array you have. HSP's protect your data ~after~ a disk failure, they prevent the nightmare scenarios of multiple disks failing in sequence due to them being from the same production lot. They've personally saved my bacon after we had a disk failure. One disk failed then the next day another one in the same array failed. If we didn't have a stand-by disk then we would of had system downtime and had to resort to older disk backups on tape.
m
0
l
June 7, 2011 7:04:42 AM

Ok RAID 0 can run in two different modes depending on your storage controller. You really need to read the manual or just ask the vender at this point.

The first is the striping that we all know and love. The second is spanning. We run our EMC SAN arrays using RAID50 spanning so that we can expand the size by adding additional RAID5 arrays onto it. You should be able to do the same but please check with your vendor for specific steps and requirements. RAID 60 spanning is what your looking for. The storage processor should be able to add another chassis to the array easily.
m
0
l
June 7, 2011 7:10:03 AM

Thanks man, your help tonight has been invaluable RAID60 spanning is what I'm looking for then. I've run through the manual a bit and it didn't go into great details. I will try to talk to one of their support staff tomorrow. I'll let you know how it turns out.

Hat's off to you man!
m
0
l
June 7, 2011 7:18:24 AM

Well some systems will do RAID50(60) striping and use dynamic redistribution. When you add a new member it will recalculate where data ~should~ go and slowly move / reshuffle it around. That goes deep into vender territory as each major vender does their own magic. Now that you know the terminology you can ask the right quests to the vender support staff and they should be able to instruct you on how it works.
m
0
l
June 7, 2011 7:28:19 AM

I looked at the Manual and it didn't go into much detail in regards to RAID 60, but he specs say: RAID 60 in addition to Automatic Array Rebuild, Array Expansion, Array Migration, Bad Sector Remapping, Dedicated Hot Spares, Hot Plug, Hot Swap.

There's no data on the arrays so I have the luxury of "playing" with it without worrying about destroying data. I will test out RAID 60 tomorrow.
m
0
l
June 7, 2011 8:00:34 PM

@palladin9479,

I spoke with promise today and they recommended against spanning the raid60 across separate physical enclosures because if either one went down it would make all data inaccessible, what are your thought on this?

Thanks
m
0
l
June 8, 2011 1:21:48 AM

Depends on the quality of the arrays and the method that their being linked. From the way you described it the second unit was acting as a drive expansion to the first, in this case they can be considered a single unit. There shouldn't be any logical reason not to treat them that way.

Try to map out the possible situations where a chassis goes down. Power being pulled, bad PSU (does it have redundant PSU's), the cable being disconnected. What kind of warranty does this have on it and how long is it for? If you split your data up and a chassis goes down you still lose access to data just not all of it. And if the main chassis goes down you lose access to everything anyway as that contains the storage processor. So really we're only discussing what happens if the expansion chassis goes offline.

Honestly while you can protect against single or even multiple disk failures, protecting against a physical enclosure is nearly impossible. The only solution is to have 4~5 enclosures and use RAID amongst them, which gets ridiculously expensive and complicated.

Also it sounds like they didn't do a good job of explaining things to you. RAID0 striping if one member goes down you lose the data, but RAID0 spanning if a member goes down you only lose the data stored on that member. Otherwise run a disk repair utility and the volume will come back online. I think they immediately assumed you were talking about RAID60 striping, or they don't support the spanning version of that implementation.
m
0
l
June 8, 2011 12:16:14 PM

In the several times I've called Promise I can't that their support is all that great. From what I understand RAID 0 spanning will span the data across sequentially so the parity isn't distributed across the span so if one had a span across two physical chassis in the even one chassis and disk were to complete fail only the data on the failed unit would be lost. If the implementation of RAID 0 supports only striping than all data could be lost? Is this correct?

So it sounds like I need to find out if their RAID60 supports RAID0 spanning or not? If not, than the RAID60 wouldn't probably not be a good option for me. In that case I'm thinking I should do two large RAID6's with two spares in each unit.

Thanks again man.
m
0
l
June 8, 2011 2:57:02 PM

I called promise and their systems do not support RAID0 Spanning only striping. I thinking I'll just go with two 22 drive raid6 with 2 dedicated spares in each chassis.
m
0
l
June 9, 2011 1:34:53 AM

If that is the case then you might want to look into mounting volume's as folders and not drive letters. NT allows you to mount a volume as a folder or subfolder instead of a drive letter. For example you could create a folder "C:\Root-Share" and share it out. Then mount one of your arrays as "C:\Root-Share\Array1" and the other as "C:\Root-Share\Array2". This should reduce the complexity of having multiple "drive letters" shared out.
m
0
l
June 11, 2011 6:00:38 AM

That's exactly what I did. I mount the shares on the server and the two volumes are transparent to the end user. In the end it makes no difference technically weather there is more than on volume on the server.

Thanks for your continued help sir:) 
m
0
l
!