- May 23, 2010
- 3
- 0
- 18,510
Hi,
I'm looking at alternatives to RAID in order to improve disk read performance for small files. Just bear with me for a minute to explain:
I want to find out if mirroring data across multiple HDDs can improve disk read performance. I'm working on an app which reads little amounts of data (<100 KB) and am looking to maximize the number of such reads per second, therefore for a typical HDD seek time would be 7-10ms and read time would be 1-2ms and RAID usually improves read time at the cost of worse seek times. Data is around 250GB and I need redundancy so a SDD solution would be too expensive. RAID doesn't help much as seek times are actually getting worse when you add more HDDs. I already thought about RAM caching but I cannot afford to go over 8-12GB of RAM as prices go up very fast over those numbers. RAM caching will definitely help a lot but problem still remains for the rest of the data.
What I'm thinking to try is to have data mirrored across multiple 250GB drives and have a load balancing reader module (at software level) that would distribute the reads across all drives. I wouldn't worry about the actual mirroring of data and how drives may get out of sync when data is updated. Let's just say data is already there. Main question is how exactly a southbridge may help with this. I worry that if you run reads in parallel on multiple HDDs at once there will be a performance bottleneck from the southbridge or transfer bus. As I'm not a hardware guy, my questions are on the lines of: can I seek on 4 drives at a time? or maybe seek on 2 drives while I read data from 2 other dives without a performance penalty?
I wrote a little app to benchmark HDD parallel random access times and throughput and I feel the results have a lot to do with the southbridge chipset I'm using. After running it a couple of times on my desktop computer (I got 3 HDDs: one IDE and two SATA) I got the following stats:
Random access times for each HDD. 2nd column shows tests run on 1 HDD at a time, 3rd column shows tests run on all HDDs at the same time (in IO op per second):
1 by 1 parallel
HDD1 181 60 (it's a WD raptor)
HDD2 117 74
HDD3 148 147
A similar test for throughput with 2nd columns showing tests run on 1 HDD at a time, 3rd column showing tests run on all HDDs at the same time (in MB/s):
1 by 1 parallel
HDD1 72 43
HDD2 60 43
HDD3 72 64
My question is why the big difference? And why HDD3 appears to perform better on parallel runs compared to the other two?
I'd say there's definitely a performance drawback as I would expect the numbers to be similar for the two scenarios (1 by 1 vs parallel). I know overall running IO operations on all drives at once is better than running them on each drive one by one, but it's not as good as I would expect. It appears parallel runs go at about 70% of the sum of 1 by 1 runs, and this is only for my particular computer.
I might add the app runs separate threads for each HDD, and random access IO are run one at a time, so the queue is always 1 for each HDD (longer queue tests might be interesting). For random access IOs the app only reads one byte (I don't know how much data the drive actually reads, but that's what I tell it to do). I run Windows XP, tests were run on 1GB files which may be a little fragmented. My Gigabyte motherboard (GA-965P-DS4) is using the ICH8R chipset with SATA II support. I don't have AHCI enabled. I rebooted the OS before every test to be sure Windows file caching is not influencing the results. I'm a software guy and got limited knowledge on how hardware works at low level.
For the final computer I was thinking to use SB850 as a south bridge and use it with a 6 core AMD for better price/performance. For financial reasons I cannot purchase the computer before I can decide wether to use RAID or my alternative idea.
Any help would be appreciated. I'll do my best to answer any questions.
Regards,
Mihai Stanimir
I'm looking at alternatives to RAID in order to improve disk read performance for small files. Just bear with me for a minute to explain:
I want to find out if mirroring data across multiple HDDs can improve disk read performance. I'm working on an app which reads little amounts of data (<100 KB) and am looking to maximize the number of such reads per second, therefore for a typical HDD seek time would be 7-10ms and read time would be 1-2ms and RAID usually improves read time at the cost of worse seek times. Data is around 250GB and I need redundancy so a SDD solution would be too expensive. RAID doesn't help much as seek times are actually getting worse when you add more HDDs. I already thought about RAM caching but I cannot afford to go over 8-12GB of RAM as prices go up very fast over those numbers. RAM caching will definitely help a lot but problem still remains for the rest of the data.
What I'm thinking to try is to have data mirrored across multiple 250GB drives and have a load balancing reader module (at software level) that would distribute the reads across all drives. I wouldn't worry about the actual mirroring of data and how drives may get out of sync when data is updated. Let's just say data is already there. Main question is how exactly a southbridge may help with this. I worry that if you run reads in parallel on multiple HDDs at once there will be a performance bottleneck from the southbridge or transfer bus. As I'm not a hardware guy, my questions are on the lines of: can I seek on 4 drives at a time? or maybe seek on 2 drives while I read data from 2 other dives without a performance penalty?
I wrote a little app to benchmark HDD parallel random access times and throughput and I feel the results have a lot to do with the southbridge chipset I'm using. After running it a couple of times on my desktop computer (I got 3 HDDs: one IDE and two SATA) I got the following stats:
Random access times for each HDD. 2nd column shows tests run on 1 HDD at a time, 3rd column shows tests run on all HDDs at the same time (in IO op per second):
1 by 1 parallel
HDD1 181 60 (it's a WD raptor)
HDD2 117 74
HDD3 148 147
A similar test for throughput with 2nd columns showing tests run on 1 HDD at a time, 3rd column showing tests run on all HDDs at the same time (in MB/s):
1 by 1 parallel
HDD1 72 43
HDD2 60 43
HDD3 72 64
My question is why the big difference? And why HDD3 appears to perform better on parallel runs compared to the other two?
I'd say there's definitely a performance drawback as I would expect the numbers to be similar for the two scenarios (1 by 1 vs parallel). I know overall running IO operations on all drives at once is better than running them on each drive one by one, but it's not as good as I would expect. It appears parallel runs go at about 70% of the sum of 1 by 1 runs, and this is only for my particular computer.
I might add the app runs separate threads for each HDD, and random access IO are run one at a time, so the queue is always 1 for each HDD (longer queue tests might be interesting). For random access IOs the app only reads one byte (I don't know how much data the drive actually reads, but that's what I tell it to do). I run Windows XP, tests were run on 1GB files which may be a little fragmented. My Gigabyte motherboard (GA-965P-DS4) is using the ICH8R chipset with SATA II support. I don't have AHCI enabled. I rebooted the OS before every test to be sure Windows file caching is not influencing the results. I'm a software guy and got limited knowledge on how hardware works at low level.
For the final computer I was thinking to use SB850 as a south bridge and use it with a 6 core AMD for better price/performance. For financial reasons I cannot purchase the computer before I can decide wether to use RAID or my alternative idea.
Any help would be appreciated. I'll do my best to answer any questions.
Regards,
Mihai Stanimir
Facebook
Twitter
Instagram