Sign in with
Sign up | Sign in
Your question

16xSSD Raid Array

Last response: in Storage
Share
March 29, 2012 4:40:55 PM

Lets say I have 16 128 GB Samsung 830 SSDs and I want to put them in a RAID 10 array.

I was looking on newegg and found this controller:
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

It looks like it only supports 8 drives though. Can I purchase multiple cards to support the additional 8 and these cards work together somehow? Or, is there a SataIII Raid card available that will support 16 drives that isn't outrageously expensive.

Essentially, I am trying to get as much performance out of these 16 drives as possible. I just need to be told how exactly to do that.

More about : 16xssd raid array

a b G Storage
March 29, 2012 6:20:22 PM

Don't put SSDs in RAID 10. You lose TRIM support and your performance is going to drop noticeably as time goes on.

If you want to spend stupid amounts of money on crazy fast storage, get a PCIe based SSD instead. You can get 1TB on such a drive for a couple thousand bucks.
m
0
l
March 29, 2012 6:51:32 PM

willard said:
Don't put SSDs in RAID 10. You lose TRIM support and your performance is going to drop noticeably as time goes on.

If you want to spend stupid amounts of money on crazy fast storage, get a PCIe based SSD instead. You can get 1TB on such a drive for a couple thousand bucks.


My understanding was that newer cards supported TRIM in a RAID setup. When you say PCIe based SSD, I think you are talking about something like the FusionIO drive? I don't believe it approaches anywhere near the throughput of what a RAID system like what I am talking about would approach. Where the FusionIO drive is around 1,000 MB read, 16x SSds would be closer to 3 or 4,000.

That is based on my limited understanding of the different technologies. Feel free to correct me if I am wrong.
m
0
l
Related resources
a b G Storage
March 29, 2012 7:09:54 PM

The newest controllers support RAID 0 only, as far as I know. This was a pretty recent development as well, just several months ago. If I'm wrong here, somebody please come in and correct me.

As far as performance, you can get way higher than 1Gbps with PCIe drives now. This one boasts close to 2 Gbps, with 245k IOPS at around $2.65/GB. 16 Samsung 830s would provide the same space for $2.81/GB, not counting the cost of the RAID controller card you'd also need to pull it off.
m
0
l
a b G Storage
March 29, 2012 7:17:21 PM

Just double checked, and currently TRIM is still a no-go in RAID. The latest alpha (not even beta yet) version of Intel's RST supports TRIM with RAID 0 only. You can also fudge it in Linux, but it's not true TRIM support.
m
0
l
March 29, 2012 7:35:02 PM

willard said:
Just double checked, and currently TRIM is still a no-go in RAID. The latest alpha (not even beta yet) version of Intel's RST supports TRIM with RAID 0 only. You can also fudge it in Linux, but it's not true TRIM support.


Is TRIM support really an issue if there are very few writes to the hard drive? We have very large databases, but they are almost exclusively read operations. Once per year we get data that is put in over the course of a day.
m
0
l
a b G Storage
March 29, 2012 7:56:02 PM

Hrmm, I don't really think I'm qualified to answer that. It seems to me that you'd probably be fine as long as you occasionally wiped the drives to reset their performance, but you might not even need to do that.

I'm really out of my element in enterprise class storage.
m
0
l
March 29, 2012 8:23:45 PM

willard said:
Hrmm, I don't really think I'm qualified to answer that. It seems to me that you'd probably be fine as long as you occasionally wiped the drives to reset their performance, but you might not even need to do that.

I'm really out of my element in enterprise class storage.


Understood. I appreciate the response anyway! Also, even though the space would be the same if I did a revodrive, I "believe" the IOPS of as many SSDs would be far superior. 8 SSD 830s would theoretically produce 640,000 IOPS compared to the linked drives 243,000.

Granted.... theoretical.
m
0
l
a b G Storage
March 29, 2012 9:02:05 PM

Yeah, I looked for benchmarks on modern SSDs in big RAID arrays and couldn't find any. Found a really old one comparing 20GB SSDs to short stroked velociraptors, but that was it. The theoretical performance of RAID is usually quite a bit higher than the actual.

I think it would be pretty safe to say, however, that 16x SSDs in RAID 10 would be measurably faster than a single powerful PCIe SSD. How much faster, I don't know, and that performance certainly comes with a price (lack of TRIM and higher $/GB).
m
0
l
a b G Storage
March 29, 2012 10:24:07 PM

I think the post regarding PCIe devices is really the right answer. For the amount of money you will spend on the controller and the drives, I think you will get better performance for the money from a 1TB PCIe device.


SLC solutions such as fusion IO will offer the best performance and will exceed that of your 16x128GB SSD array by a good margin. The enterprise solutions usually have some type of parity/redundacy built into the board themselves so a 1TB board might actually have 1.3TB worth of cells to account for cell usage/problems etc.

Read Bandwidth (1 MB)
3.0 GB/s

Write Bandwidth (1 MB)
2.6 GB/s

Read IOPS (Seq. 512 B)
702,000

Write IOPS (Seq. 512 B)
937,000

Read Access Latency
47µs

Write Access Latency
15µs

Bus Interface
PCI-Express 2.0 x8

Weight
Less than 11 ounces
m
0
l
a c 99 G Storage
March 29, 2012 10:34:41 PM

At some point, you will surpass the amount of MBps the the CPU can handle. I don't know what that number is, but 16 SSDs could get you 8,400MBps reads.

But why? I can't even tell the difference between 2 SATA II SSDs in RAID 0 and 2 SATA III SSDs in RAID 0. Each PC boots in about the same time, and programs launch equally fast.

Yes, the "benchmarks" show that my SATA III array is twice as faster (even more) than my SATA II array. But my eye can't tell the difference.

You're looking at over $2000, plus the cost of RAID controllers, to yelid what?
m
0
l
a b G Storage
March 29, 2012 10:39:02 PM

I agree you can't tell them difference for consumer/home purposes, I'm not sure I could even tell the difference between a single fast SSD and 2 in RAID 0. I think th eOP is looking for a database server solution. Depending the database you very well might need to go beyond 2 SSDs worth of IOPS. The OP is more interested in IOPS than raw throughput
m
0
l
March 30, 2012 2:02:19 PM

foscooter said:

You're looking at over $2000, plus the cost of RAID controllers, to yelid what?


Tokencode hit the nail on the head. I need to create a DB that returns complex queries as quickly as possible. Currently my development server is a simple machine. A single momentus hybrid hd, 16 GB of Ram and 2 quad core intel processors.

I can't say money isn't an issue, but spending 4 or 5 thousand dollars is not a problem at all to build a database with minimal time between the time the query is executed by the web server and the data is returned.

I was planning on putting the log files for the DB on a more traditional hd, a set of momentus drives, to avoid writing onto the SSDs as much as possible.
m
0
l
March 30, 2012 2:08:41 PM

DelroyMonjo said:
I found this extremely interesting and it addresses SSD:
http://en.wikipedia.org/wiki/Nested_RAID_levels


Thanks for the link. I've been trying to find some realistic benchmarks. I thought surely someone has done some extensive testing with SQL and SSD Raid Arrays, but I haven't found a whole lot.

Usually the articles will just give a generic statement that "yes, it is faster", but I would like to be able to find an answer to the following question.

"If my current query takes .8 seconds to run on a single drive, how long would it take to run on a raid of 4 (or more) SSD drives?"
m
0
l
a b G Storage
March 30, 2012 5:22:03 PM

There are a lot of DBs being run on enterprise class SSD arrays but they are probably under the name of the SAN that the SSD array is plugged into such as Compellant, EMC, etc... Usually those using SSD arrays because they need the speed also need them to be EXTREMELY reliable so they do not build arrays of consumer grade drives but opt to go with a supported solution from a vender although that is far more expensive. If you're running on a single hybride drive now, how do you know you need that many IOPS? Have you tried a single dedicated SSD? It's impossible to answer your question which why you won't find any answer to it. The total time it takes to execute a query depends on many factors such as processors, memory, not just the disks. Let's say the drives are twice as fast, and everything else is the same. If you query consists of 20% reads, 50%cpu and 50% of the data is in ram then your query might only show a real-world 5% improvement in total time because the disks are only being used for 1/2 the 20% of the reads. All the CPU and th portion of the data in memory would remain the same.

Assuming this is MS SQL, use Query Analyzer to determine what percentage of the query is reads and use the size of the DB and amount of rame to estimate the amount of the data in memory.
m
0
l
a c 304 G Storage
March 30, 2012 7:45:50 PM

Ehh, Tom, there is a pretty good chance that your CPU will become the limiting factor. It may be throttled by one HDD, but if you keep increasing your random IO rate you may find an upper bound. In which case, you would have to start building a farm for further improvements, where queries are distributed among multiple machines with the same data.

It's easy enough to test. Modify the app to use random data (cutting out the disk part) and see if you can saturate the server with your expected workload.

Have fun.
m
0
l
!