We were in the middle of setting up a new storage test system to replace our aged platform when we noticed something really strange: Intel’s top-of-the-line X25-E flash solid state disk (SSD) delivered less throughput on the new test system than it did on the old one. It took us a few days to isolate the reason, and our findings are noteworthy, as they illustrate the first practical bottleneck for flash SSDs. Mainly, they cannot provide their maximum performance if you take advantage of processor and platform power saving features.
We further explored our findings by enabling and disabling various power saving options, using two X58 motherboards from Gigabyte and Supermicro.
Updating The Storage Testing Platform
We’ve been using our storage reference platform for more than four years, as it has allowed us to compare test results of drives and controllers since the introduction of the system. All hard drive tests, such as the recent budget drive roundup, our short stroking analysis or some testing on acoustic management, together with all tests for our 3.5” Desktop HDD charts, 2.5” Notebook HDD charts, Flash SSD charts and Enterprise HDD charts, were conducted on this platform. It was based on an Intel Xeon Nocona single-core processor with 3.6 GHz and 1 MB L2 cache (Prescott generation) and the E7520 server platform, which was the first to offer PCI-X and PCI Express. Both the platform and the storage controller—a Promise SATA 300TX4 controller—are rather outdated now, so we decided to switch to a Supermicro X8SAX and a Core i7 920 processor.
The Issue: Performance Numbers Were Off
The new system is fast, and it comes with the latest Intel ICH10R southbridge. But its throughput using our fastest SATA drive—an Intel X25-E enterprise flash SSD—was lower than on our old storage system. Instead of reaching 200 MB/s, the new Core i7 system was limited to 182 MB/s.
At this point, we have to underscore that the X25-E is capable of delivering 230 MB/s, and it was only limited to 200 MB/s on our prior storage reference system by the old controller. Since all of our tests were performed on this controller, results were comparable at all times, but this performance drop made us scratch our heads.
Isolating The Performance Break
A BIOS update by Supermicro did not solve the issue, so we began our analysis by trying a second motherboard, which was a Gigabyte EX58-UD4P, so we could see whether or not the issue was related to the Supermicro motherboard. This wasn’t the cause, as the throughput on Gigabyte’s X58 motherboard was also somewhat bottlenecked. We measured a maximum read throughput of 186 MB/s; better, but still miles away from the expected 230 MB/s maximum. So we went on and disabled all unwanted components and BIOS features, until we were able to find the culprit: the bottleneck begins as soon as you start using processor and platform power saving features.
Articles like this make me wish we could rate articles. Great article!
how bout amd???
Next step is to find a workaround.
I don't yet have a system to experiment with but have a few suggestions.
How about setting process affinity and raising priority of the processes involved with disk access?
On my Dell laptop, I use an application called SpeedSwitchXP that makes it possible to manipulate the C-state policies for Demote/Promote-limits on microsecond level. It seems reasonable that introducing a little more hysteresis in the C-state policies would keep the CPU faster between disk operations.
Run 4 & 5 for X8SAX show the same stuff disabled (and a capitalization missed) .... mistake?
Do power saving options also affect hard drive performance?
No. The fastest hard drives available are still far from reaching 200 MB/s. We benchmarked two 15,000 RPM SAS drives, but could not see a difference as significant as was the case with the Intel X25-E flash SSD.
Btw, does this assessment count for raid usage as well? I'm seeing fairly irregular speeds on my raid 5 (5x500gb) on my ich9r based system. Gigabyte said it was because I was using the raid as a system drive. So I created a raid 0 of 2x500gb drives for windows, but the irregular speeds on the raid 5 remained.
Just out of curiosity, will the performance be affected if a dedicated raid card is used to connect the SSD?
@cedriclam: Hard to say, but I'd guess yes. We will look into this again soon, using different platforms.
We didn't have two X25-Es to check if this is also the case for RAID setups, but my feeling says yes. The same applies to other controllers, but we'll look into this in greater detail soon!
This may be a case of benchmarks not showing what true real world results would be. Since most things that would truly stress the hard drive performance would most likely also be using more CPU so they would not be in the low power states.
Just thinking out loud.
Tomshardware dudes:

Today I feel a deep rant, giving you 5 stars for this investigation.
Keep doing good work.
Nice work on the article! I've got a couple of questions though:
Does this affect I/O performance as well, or is it just throughput?
and,
Do you get the same affect on AMD systems?
patrick, what car are you driving in your avitar? (just curious)
Does the SSDs support Write Caching (in disk properties), and if so, have you turned it off? It's been my experience with regular HDDs that turning off write caching has an impact on CPU usage...
how bout amd???
For god sake!!! I own an Opteron, but please you AMD spammers are killing us! Who the hell would buy a crappy slow $200 Phenom to pair it up with a $800 SSD?
Buy a Tata $2500 car and put some 26in wheels on it too!
For god sake!!! I own an Opteron, but please you AMD spammers are killing us! Who the hell would buy a crappy slow $200 Phenom to pair it up with a $800 SSD?Buy a Tata $2500 car and put some 26in wheels on it too!
As far as my knowledge goes, Phenoms II are not slow. ;-) Sure they are slower than core i7, but they are fairly cheap. Not only core i7 can benefit from SSD.
I'm sure Tom's will test this issue on an AMD platform too. :-)
Do MLC flash drives also exhibit the same slowdown with the power saving features turned on?
I know they are much slower to start with, but do they also get affected equally?
... Who the hell would buy a crappy slow $200 Phenom to pair it up with a $800 SSD?Buy a Tata $2500 car and put some 26in wheels on it too!
Somebody who need the fastest SSD, on a limited budget, and found that AMD does not have the energy-saving performance issue (Who knows?. for THAT reason we want to see the AMD benchmarks)
please benchmark some of the new OCZ SSD's now that you have a new reference system, i've been dying to see real world numbers for the OCZ apex and especially the VERTEX series! Two drives that claim decent performance without completely emptying my wallet.
In the top paragraph of the "Outlook, Conclusion And Remaining Questions & Answers" section the second sentence reads: "...and the X25-E (for consumers)" That should probably be X25-M
Otherwise, very interesting article.
-mcg