Benchmarking For The Enterprise: A Whole New World
Just as server- and workstation-oriented processors have to be tested differently than desktop CPUs, so too does enterprise-oriented storage need to be evaluated in a unique way.
We got the above slide from last year's Flash memory Summit in Santa Clara, CA. The assumption for enterprise storage is that it's generally full, and often being accessed around the clock. As a result, fresh-out-of-box/burst performance is largely irrelevant. There's little to no idle time for the background garbage collection and TRIM commands to recover performance, which means an SSD in such a taxing environment is going to hit its steady-state and stay there.
Ultimately, we have to use a different methodology to get to and then test steady-state performance. The goal is to benchmark at a point where an SSD's performance no longer changes over time, necessitating constant writes in order to determine sustained performance. The chart above illustrates how, after some period of use, an SSD drops from its out-of-box performance level to a more sustainable steady-state level.
In order to attain that second point, we precondition our SSDs before running our enterprise benchmarks. But because every drive's steady-state point is different (and because there are multiple steady states, depending on the workload you run), we specifically subject our SSDs to two types of conditioning:
- For our 4 KB random, database, file server, and Web server tests, we write 3x full capacity of the drive using random writes.
- For our 128 KB sequential tests, we write 3x full capacity of the drive sequentially.
