Results: Tom's Hardware Storage Bench v1.0
Storage Bench v1.0 (Background Info)
Our Storage Bench incorporates all of the I/O from a trace recorded over two weeks. The process of replaying this sequence to capture performance gives us a bunch of numbers that aren't really intuitive at first glance. Most idle time gets expunged, leaving only the time that each benchmarked drive was actually busy working on host commands. So, by taking the ratio of that busy time and the the amount of data exchanged during the trace, we arrive at an average data rate (in MB/s) metric we can use to compare drives.
It's not quite a perfect system. The original trace captures the TRIM command in transit, but since the trace is played on a drive without a file system, TRIM wouldn't work even if it were sent during the trace replay (which, sadly, it isn't). Still, trace testing is a great way to capture periods of actual storage activity, a great companion to synthetic testing like Iometer.
Incompressible Data and Storage Bench v1.0
Also worth noting is the fact that our trace testing pushes incompressible data through the system's buffers to the drive getting benchmarked. So, when the trace replay plays back write activity, it's writing largely incompressible data. If we run our storage bench on a SandForce-based SSD, we can monitor the SMART attributes for a bit more insight.
| Mushkin Chronos Deluxe 120 GBSMART Attributes | RAW Value Increase |
|---|---|
| #242 Host Reads (in GB) | 84 GB |
| #241 Host Writes (in GB) | 142 GB |
| #233 Compressed NAND Writes (in GB) | 149 GB |
Host reads are greatly outstripped by host writes to be sure. That's all baked into the trace. But with SandForce's inline deduplication/compression, you'd expect that the amount of information written to flash would be less than the host writes (unless the data is mostly incompressible, of course). For every 1 GB the host asked to be written, Mushkin's drive is forced to write 1.05 GB.
If our trace replay was just writing easy-to-compress zeros out of the buffer, we'd see writes to NAND as a fraction of host writes. This puts the tested drives on a more equal footing, regardless of the controller's ability to compress data on the fly.
Average Data Rate
The Storage Bench trace generates more than 140 GB worth of writes during testing. Obviously, this tends to penalize drives smaller than 180 GB and reward those with more than 256 GB of capacity.
The Ultra Plus drives don't mess around in this metric. A combination of fast flash and nCache help the 64 and 128 GB models tremendously. Given the basic performance data implications, we know these drives are really flying at lower queue depths. While the trace does spike up to higher queue depths at times, the outstanding commands fall between one and three on average.
Had this trace comprised fewer writes, the 64 GB Ultra Plus would look even better. But almost 150 GB of writes quickly put smaller drives into a coma, forcing them to struggle with garbage collection routines early on in the benchmark. You can see this in the average service times, where write requests require additional overhead in this state.
Unfortunately, busy time and the MB/s stat you generate with it aren't adept at measuring higher queue depth performance. Despite the corner case testing that shows one drive far ahead of another at low queue depths, all of the SSDs we're testing are basically the same in the background I/O generated by Windows, your Web browser, or most other mainstream applications. Only when we're presented with lots of data throughput (reads and writes) in a short time do we see separation between high-end and mid-range drives. For that, we need another metric.
Service Times and Standard Deviation
There is a wealth of information we can collect with Tom's Storage Bench above and beyond the average data rate. Mean (average) service times show what responsiveness is like on an average I/O during the trace. It would be difficult to plot the 10 million I/Os that make up our test, so looking at the average time to service an I/O makes more sense. We can also plot the standard deviation against mean service time. That way, drives with quicker and more consistent service plot toward the origin (lower numbers are better here).
The three Ultra Pluses demonstrate admirable service times. On average, they handle I/O requests without much latency. The 256 GB model averages just 440 microseconds over the 10 million I/O requests that make up our trace. Most of those are 4 KB transactions, but a significant number are larger transfers. The 128 GB drive isn't far behind. The 64 GB Ultra Plus looks pretty rough in comparison, but its average service times are just slightly behind Intel's 120 GB SSD 525. The 60 GB SSD 525 is so far out of this chart's range that we'd have to expand its scale to ridiculous proportions.
The statistical measurement of standard deviation can indicate a drive's consistency throughout the trace playback. SanDisk's Ultra Pluses are ever so slightly less consistent than the competition, which isn't unreasonable to assume since the 88SS9175 controller is working with half as many channels. The thing is, our trace pounds 64/128 GB-class drives into submission. Even if that wasn't true, the smaller capacities would still be at a disadvantage. The fact that we're still seeing not just passable, but excellent performance is most welcome indeed.
