Enter the Storage Performance Council
Enterprise SSDs: How to Prove Real World Performance
What's this
The SPC is a non-profit organization comprised of over four dozen companies from across the storage industry. Their mission is simple: provide the world with a reliable, consistent toolset for evaluating storage performance and, as the group’s site notes, “serve as the catalyst for performance improvement in storage subsystems.” The SPC strives toward these goals with a series of different benchmarks, each targeting a different combination of workload type and storage target, and measurement metric as shown below.
For the purposes of evaluating drive performance, the most appropriate test is SPC-1C. This benchmark’s methodology (as detailed in the SPC’s specification) is quite complex and thorough. It is comprised of multiple types of data divided into different types of units that target drives in various ways. Specifically, the SPC-1C benchmark uses Business Scaling Units (BSUs) as its core workload element. One BSU represents the load of five I/Os per second generated by a group of users. Furthermore, SPC-1C divides the total Application Storage Unit (ASU) capacity of the drive, its maximum addressable space for read/write operations, into three divisions: 45% into ASU-1 for raw incoming data, 45% into ASU-2 principally for user- or application-generated data, and 10% into ASU-3 for log files used for data integrity checking. A complex set of parameters governs how I/O values are created within the SPC-1C Workload Generator. These values, in turn, are doled out in a regulated I/O flow to the ASUs. For example, ASU-1 receives four simultaneous streams: a randomized read and a randomized write stream spread over the entire ASU, a sequential read stream, and another I/O stream focused on a particular area within the ASU. All told, these four streams account for 59.6% of the total SPC-1C workload.
Source: SPC Benchmark 1C Official Specification (http://bit.ly/IEDWSd)
The following graphic illustrates how a conventional benchmark will blanket reads and writes all over an SSD while SPC-1C writes in patterns much more in line with how real world applications are likely to address a drive.
Another facet of consistency is the persistence of data integrity even in the face of power loss. The SPC-1C Workload Generator can write and log random 16-block I/O requests across the drive’s addressable area. The test system is then shut down and rebooted. The Workload Generator uses the log file to verify that each block written matches what was originally written before power cycling.
Drive specialists also understand that performance characteristics can change both during ramp up/shut down phases as well as at different load levels. An SSD capable of 20,000 IOPS may well perform differently under a 15,000 IOPS load than it does at 10,000 IOPS. Data center buyers should be aware of these variances, but most tests fail to take such variables into account. SPC-1C assesses different load levels as seen in the following graphic:
Source: SPC Benchmark 1C Official Specification (http://bit.ly/IEDWSd)
This full-spectrum is doubly important since it’s common for storage devices not to operate at peak levels constantly, and performance anomalies may appear at lower use levels.

