What's Important: Steady-State Performance
Steady-State Performance
SSD manufacturers prefer that we benchmark drives the way they behave fresh out of the box because solid-state drives slow down once you start using them. If you give an SSD enough time, though, it will reach a steady-state performance level. At that point, its benchmark results reflect more consistent long-term use. In general, reads are a little faster, writes are slower, and erase cycles happen as slow as you'll ever see from the drive.
We want to move away from benchmarking SSDs fresh out of the box whenever possible because you only really get that performance for a limited time. After that, you end up with steady-state performance until you perform a secure erase and start all over again. Now, we don't know about you, but we don't reformat our production workstations every week. So, while performance right out of the box is an interesting metric, it's not nearly as relevant in the grand scheme of things. Steady-state performance is what ultimately matters.
While this is a new move for us, IT professionals have long used this approach to evaluate SSDs. That's why the consortium of producers and consumers of storage products, Storage Networking Industry Association (SNIA), recommends benchmarking steady-state performance. It's really the only way to examine the true performance of an SSD in a way that represents what you'll actually see over time.
There are multiple ways to get there, but we're going to use Intel’s IPEAK (Intel Performance Evaluation and Analysis Kit). This is a trace-based benchmark, which means that we're using an I/O recording to measure relative performance. Our trace, which we're dubbing Storage Bench v1.0, comes from a two-week recording of my own personal machine, and it captures the level of I/O that you would see during the first two weeks of setting up a computer.
Installation includes:
- Games like Call of Duty: Modern Warfare 2, Crysis 2, and Civilization V
- Microsoft Office 2010 Professional Plus
- Firefox
- VMware
- Adobe Photoshop CS5
- Various Canon and HP Printer Utilities
- LCD Calibration Tools: ColorEyes, i1Match
- General Utility Software: WinZip, Adobe Acrobat Reader, WinRAR, Skype
- Development Tools: Andriod SDK, iOS SDK, and Bloodshed
- Multimedia Software: iTunes, VLC
The I/O workload is somewhat moderate. I read the news, browse the Web for information, read several white papers, occasionally compile code, run gaming benchmarks, and calibrate monitors. On a daily basis, I edit photos, upload them to our corporate server, write articles in Word, and perform research across multiple Firefox windows.
The following are stats on the two-week trace of my personal workstation:
| Statistics | Storage Bench v1.0 |
|---|---|
| Read Operations | 7 408 938 |
| Write Operations | 3 061 162 |
| Data Read | 84.27 GB |
| Data Written | 142.19 GB |
| Max Queue Depth | 452 |
According to the stats, I'm writing more data than I'm reading over the course of two weeks. However, this needs to put into context. Remember that the trace includes the I/O activity of setting up the computer. A lot of this information is considered one-touch, since it isn't accessed repeatedly. If we exclude the first few hours of my trace, the amount of data written drops by over 50%. So on a day-to-day basis, my usage pattern evens out to include a fairly balanced mix of read and writes (~8-10 GB/day). That seems pretty typical for the average desktop user, though this number is expected to favor reads among the folks consuming streaming media on a larger and more frequent basis.
On a separate note, we specifically avoided creating a really big trace by installing multiple things over the course of a few hours, because that really doesn't capture real-world use. As Intel points out, traces of this nature are largely contrived because they don't take into account idle garbage collection, which has a tangible effect on performance (more on that later).
