Another question on RAID (software)

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

This one is more out of curiosity than anything else...

On Windows 2000, we are restricted to equal sizes for all
the RAIDed partitions. However, if we have a drive X that
is, say, twice as fast as drive Y, then you wouldn't get
much benefit from it (in the extreme, if one of the drives
is more than twice as fast as the other one, then one gets
higher performance NOT using RAID).

This would not be true if we were able to specify different
sizes -- the optimal point would be when the sizes are
proportional to the relative speeds (i.e., if drive X is
f times faster than drive Y, then the partition on drive
X should be f times larger, and all the data should be
split in blocks that are f times larger for drive X).

Are there systems that do this? (software or hardware?)

Or they don't even bother, given that if you're using RAID
it is assumed that you are not playing with cheap or old
stuff?

Thanks,

Carlos
--

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

Curious George wrote:

> All raid (HW & SW) limits size by the smallest drive in the array.
> Unused space can be reclaimed by making a separate logical drive or
> JBOD volume. All raid (HW & SW) limit performance by the slowest
> drive in the array.
>
> If I understand what you are asking, these are not possible. It would
> be ridiculously complicated & expensive to design a controller that
> would handle mismatched sizes and it wouldn't even work for all the
> typical raid levels.

Fair enough.

I just thought it might be neat (in the way of flexibility),
but wasn't really too sure that it would be available. The
reasons you give against the availability of such thing are
solid enough (it has been pointed out to me that the issue
of RAIDing gets fairly complicated when you have multiple
concurrent accesses, blocks of various sizes, etc. -- the
idea I was entertaining would make things exponentially more
complex, I guess).

One detail, though:

> Furthermore it is not possible to not have a
> slower disk limit the performance of an array.

No, it is possible.

The equation we would have to come to terms with is the fact
that if you have two drives, one with speed S and one with
speed f*S (with 0 < f <= 1), the theoretical maximum speed
you can get (with RAID 0, that is) is S*(1+f). If f is as
low as 0.25, then you only get an effective rate that is 1.25
the speed of the faster disk.

In a sense, the slower disk is not "holding back" the total
speed -- it's only holding back the potential for increase.
The way RAID is, a sufficiently slow disk can make the total
speed lower than the speed of one drive (i.e., it can make
it slower to use RAID0 than not to use it)

Thanks for your comments!

Carlos
--

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

On Tue, 05 Apr 2005 08:17:18 -0400, Carlos Moreno
<moreno_at_mochima_dot_com@xx.xxx> wrote:

>Curious George wrote:
>
>>>- the
>>>idea I was entertaining would make things exponentially more
>>>complex, I guess).
>>
>> It also requires designing new/nonstandard raid levels. You can't
>> have a 2 disk mirror
>
>Oh no, of course. The thing came to mind as an option for
>RAID 0 exclusively.

Ah ha. But you originally did not specify a raid level. Win2k can do
raid 0,1, & 5. Using a combination of firmware & win2k raid you can
also do nested levels.

I see what you're saying now in the OP. Your recent responses make me
interpret the OP very differently.

>>>>Furthermore it is not possible to not have a
>>>>slower disk limit the performance of an array.
>>>
>>>No, it is possible.
>>>
>>>The equation we would have to come to terms with is the fact
>>>that if you have two drives, one with speed S and one with
>>>speed f*S (with 0 < f <= 1), the theoretical maximum speed
>>>you can get (with RAID 0, that is) is S*(1+f).
>>
>> What exactly do you mean by speed? STR?
>
>Yes, sustained transfer rate would be what makes most sense
>for the purpose of my analysis.
>
>> Don't know where you got
>> this formula or what you are calculating exactly.
>>
>>
>>>If f is as low as 0.25,
>>>then you only get an effective rate that is 1.25
>>>the speed of the faster disk.
>>
>> That sounds like the slower disk speeds up the faster disk. I think
>> you mean 1.25x the speed of the slower drive
>
>No, it is 1.25x the speed of the faster.

Maybe I'm just too dense or tired at the moment, but I just don't get
it.

>If you have disk 1 with speed S and disk 2 with speed S/2
>(as an example), what the striping system would do with a
>block of length L is as follows: send a block (a stripe)
>of length 2*L/3 to disk 1, and a stripe of length L/3 to
>disk 2 --

not necessarily. Ideally RAID0 is supposed to split I/O operations
into equal-sized blocks and spread them evenly across disks.
Controllers aren't smart enough to determine relative STR and adjust
I/O accordingly. They also do not allow you to control the striping
on a low level like you are proposing. If they did you would end up
not being able to use all the disk space and would just be asking for
performance problems & support calls. You are proposing a hopelessly
complex & limiting hypothetical product.

>since disk 1 is twice as fast, they will both
>take the exact time to complete the write (theoretically,
>that is). But then, you took 2/3 of the time as disk 1
>would have taken to store the complete block, right? So,
>you're going 3/2 (1.5x) the speed of the faster.

K. But this is a 3 disk example used to counter my disagreement about
a 2 disk raid0. I'd be more convinced if you contradicted my points
directly or brought in some corroborating evidence. Finally, although
the read or write may be in parts over more than one disk it is still
a single read or write. AFAIK a write complete, for example, is
issued after everything is done. Its not like some other kinds of
packetized streaming transports.

>If disk 2 is 10 times slower, then you send 10/11th of
>the data to disk 1 and 1/11th to disk 2 -- again, you're
>in perfect balance, since both disks will take the same
>time to complete the operation; the operation is taking
>10/11 the time it would take disk 1 alone, hence we're
>going 11/10 (1.1x) faster than the faster drive.

Again you're talking about a freak accident data pattern or freak
mutant configuration not haw raid is normally conceptualized &
implemented.

Saying the maximal boost is X times N disk according to these factors
is misleading since you are talking about a rare, haphazard occurrence
when things are in balance or vaporware rather than something you
could count on & measure like max STR, burst rate, etc.

>It does work, with the nice detail that the slower disk
>does not limit the effective (total) speed, as happens
>with the actual RAID0 technique (if we were to use disks
>with different speeds)

That requires a level of intelligence or flexibility raid0
implementations just don't have AFAIK.

>>>The way RAID is, a sufficiently slow disk can make the total
>>>speed lower than the speed of one drive (i.e., it can make
>>>it slower to use RAID0 than not to use it)
>>
>> Not following. Which drive? the slower one? Sounds like you're
>> contradicting yourself either way.
>
>No, not contradicting: notice that here I was referring
>now to the way RAID 0 *is* (as opposed to what I was
>thinking it could be).

I though we were talking about what it is. Isn't that the point? If
you want to design your own RAID0 variant lets call this "Moreno
Striping" or something like that - not RAID0. If this is just your
imagination/hypothesis then don't correct me like your pipe-dream is
how things actually work.

>If you have a disk that is 10 times slower, and you still
>stripe the data into equal sizes, then disk 1 will take
>a 10th of the time to complete the operation, but you'll
>still have to wait 10 times longer until the second disk
>completes it. IOW, if you had not put the second disk,
>you would go faster.

>This does not happen if you stripe
>the data into lengths proportional to the relative speed
>of the corresponding disk.

??? A very curious proposition. What products do this? Who ever
proposed this an a respected paper?

>Once again, your argument still stands: the complexity
>does not justify th gain, mainly when we take into account
>that the cost of "getting serious" and using good and
>balanced drives is not as high :-)

>(plus, yes, this
>idea applies to RAID 0 exclusively)

in extremely rare circumstances or a theoretical mutant configuration.
purely theoretical maximal gain, not anything your could realistically
count on or expect.

If you want to continue explaining your hypothesis please provide some
corroborating evidence. RAID0 is not normally designed or discussed
that way. I've never hear of anyone designing the ability to do what
you are proposing. Maybe it's because I'm too tired, but AFAIK the
I/O is supposed to be done simultaneously & there is a single
completion point before the data can be sent out of the disk subsystem
so the slower spindle or most delayed seek will always tie up a read
or write complete. This factors relative STR out not in.

If I'm wrong, please provide some corroborating evidence; a link, a
book, a whitepaper - anything. Always eager to learn.

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

On Wed, 06 Apr 2005 10:31:43 -0400, Carlos Moreno
<moreno_at_mochima_dot_com@xx.xxx> wrote:

<snip>
>Now, let's call my idea RAID-M (M for Moreno :-)), so that we
>avoid the mix-up in the explanations...

Good Idea

>Let's compare the performance of RAID-0 and RAID-M (assuming
>that we disregard any issues with the complexity of an actual
>implementation) when we have two disks of different speeds

>(and again, we assume a rather constant value for the STR as
>a measure of "speed" ).

Which it isn't. So, for example, on the outer rings the raid requires
1 stripeisk ratio while that would have to change incrementally
somehow untill you reach the inner rings which would likely require a
very different stripe ratio.

This would be even more evident with different sized drives (or rather
drives based on different platter size/density). Actually even with
drives of the same size but different speed this would also be a
problem because one is getting assigned more chunks than the other so
the logical disk will end in different places on the physical disks
which have no relation to each other performance wise.

<snip>

>Think of it as a "load balancing" process -- by striping the
>data into lengths proportional to the relative speed, you
>avoid the condition of having one disk idle because it has to
>wait for the other one to finish. The "optimal" point, where
>load balancing occurs, is the point in which both hard disks
>take the exact same amount of time in writing their stripes
>of data (which is why in RAID-M you would stripe the data
>into blocks of lengths proportional to the relative speeds
>of the disks)

Right. I got that point from the beginning but it would take a
different kind of storage technology to be able to do this even in
theory.

>Of course, the load balancing happens automatically in RAID-0
>provided that the two disks have the same speed! The alleged
>advantage of RAID-M would be the increased flexibility that
>it allows you to use disks with different speeds, and it makes
>the most of it (where "the most" in this case means that you're
>not "held back" by a slow disk).

In theory. But also in theory I think there would need to be many
restrictions on what could be used this way. "Flexability" &
"optimization" are marginal at best in this context.

>Notice the interesting detail:
>
>In RAID-0, the effective speed of the array is always twice
>the speed of the slower disk -- it doesn't help if you have
>the faster disk being 20 times faster; the speed is always
>determined/limited by the slower disk.

Which is what I was saying all along.

>In RAID-M, the effective speed of the array is necessarily
>greater than or equal to the speed of the faster disk (the
>"equal" part is only achieved if the slower disk has speed
>0, in which case its stripe has length zero, and thus the
>speed is simply the speed of the faster drive).

Now I understand what you meant.

<snip>


Thanks for the clarification. It took me a few posts to catch on & it
really took this last one to fully appreciate your argument. I'm used
to the more typical raid questions, which this sounded like at first,
but ended up being a whole lot more interesting. Even though I
understand your argument better, I don't think it really changes my
assessment of RAID-M. I'm not just concerned about the practicality
or cost of RAID-M but the theoretical basis also.

The main problem with even the theoretical basis of raid-M is that
disk performance isn't a single variable i.e. it doesn't get distilled
down to a single measurement very well. Serial or "sustained"
transfer rate decays from outside to inside the platter at very
different rates among different model drives. Also different drives
handle random I/O very differently as well.

Your numbers work so nicely ONLY because you are restricting the
formula to look at an idealized/bastardized version of "sustained
transfer rate" exclusively and hand picking nice performance
relationships between drives. How would raid-M work with, say 3
different model drives? RAID-M with 2 drives of the same size but
different speed would also mean you would be wasting space on one of
the drives (not very desirable). If you're going to plan the matching
of sets of drives or restrict RAID-M to two drives then there is
little incentive to overcome the design hurdles when normal raid-0
performs better and is ultimately more cost effective & mature. More
importantly it would significantly inhibit the flexibility you are
looking & which is the whole point of RAID-M.

The other problem is that even if drive selection is optimal/balanced
you would have to measure & match the drives performance attributes as
well as space utilization and stripe distribution with a tremendous
level of precision. Precise performance measurement & matching is not
possible because of that first point I mentioned. Also because the
point of RAID-M is to be able to optimize the use of spare parts you
have by random occurrence, you wouldn't likely be using products with
nice performance relationships.

I also would think setup would be the opposite of what you propose;
the user would select the stripe ratio based on a benchmark and the
firmware would determine the useable disk space. It would be easier
on the end-user & designer then to have the end user calculate
relative space (remember you're not likely to have nice numbers like
2:1 or 1.25:1 and there is often confusion about space measurements)
and have the controller check that the user selection is viable and
then orchestrate the stripes accordingly. There would be more room
for error. Finally, the idea that you could do RAID-M with a single
stripe ratio is not viable with disk technology even in theory. The
idea might be better suited to other technologies.

RAID-M is an interesting concept. While raid0 is simple relative to
other levels, it is not that simple from a performance or performance
optimization perspective. This is because we are trying to make the
disks work in concert and to conduct truly simultaneous I/O. Latency
and spindle sync are issues because they eat away at the ability for
I/O to be simultaneous and therefore also expected gains. Spindle
syncronization is obsolete at this point partly due to faster spindle
speeds and also due to masking techniques in the drive & controllers
firmware like caching & coalescing etc. But I can recall specifically
an array of Quantum Atlas V's (7200 RPM) I resurrected from basically
a dog to really quite peppy with spindle sync. I know there were some
companies that were also doing this a few years back, taking older,
cheaper, slower drives and using spindle sync to make the array appear
better than it was to increase profits.

Even two similar performing but not identical drives reveal out-of
sync problems in part because specs like 7200 or 10K or 15K rpm are
not really exact figures (just like disk space) and also due to
different firmware approaches to handling I/O. So two very different
drives have a more noticeable problem working together and that would
even further eat away at performance.

It would be nice if disk performance could be reduced to a single,
easily measurable number, and that there could be such flexibility as
raid-M requires. But there is another component your model does not
take into account; firmware compatibility. While firmware can
optimize performance in both single and normal raid configurations,
there are limitations to the ability of some products to perform in
raid that have historically required matching moth model & firmware in
arrays. In fact disk firmware may even require tuning to work
properly for each raid-m proportional scenario (just as they do for
normal raid as apposed to vanilla disks) or might simply perform
atrociously as they are in real world RAID-M.

For example the first WD 80 gig PATA drives, which were originally not
intended for raid, performed horribly in raid including simple
striping. This required firmware correction both by WD (upgrades of
course not availabvle to users) and avoidance of many controllers.
Back in the days of Mylex, you had to be even more careful with
firmware versions. Conflicting versions or some combinations of
controllers & disk firmware could actually cause damage to the drives.
I recall an array of Quantum/Maxtor 10K3's that were getting PFA
errors (i.e. SMART failure) even though the drives were basically
good- because of this. Between these problems and the high infant
morality rates and a batch of bad motors, the array ended up not being
viable.

I'm glad you've told me more of where you're coming from. I hope I
have done the same and not simply been repetitive. It's truly an
interesting concept. Unfortunately for performance the simplest
solution usually wins out IMHO. Maybe you could try designing
something like this for flash or ram?


Cheers.

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

Curious George wrote:

> Thanks for the clarification. It took me a few posts to catch on & it
> really took this last one to fully appreciate your argument. I'm used
> to the more typical raid questions, which this sounded like at first,
> but ended up being a whole lot more interesting. Even though I
> understand your argument better, I don't think it really changes my
> assessment of RAID-M. I'm not just concerned about the practicality
> or cost of RAID-M but the theoretical basis also.
> [...]

I'm equally grateful for the expanded analysis; you're right
that I had overlooked the variations and the difficulty to
summarize disk performace into a single, nice parameter.

Not that I thought that you were opposed to the idea because
you didn't understand it -- it was clear to me from your very
first reply that your argument was solid; I'm glad that I
could clarify the details about my OP, since that allowed for
a more specific, more interesting analysis of why the idea is
not as sound as it may seem following a light/naive analysis.

Thanks,

Carlos
--