Conclusion
Simply replacing the Adaptec RAID 5805 cards with the brand new product generation from LSI makes it clear that the RAID controller was a real bottleneck for our 16 SSD armada. Granted, the Adaptec cards are still feature-rich and balanced enterprise-class SAS HBAs, but the LSI MegaRAID 9260-8i and 9210-8i (aka Intel RS2BL080) are the first storage cards to utilize PCI Express 2.0 rather than 1.1, which doubles the bandwidth of the x8 slot from 2 GB/s to 4 GB/s. In addition, both cards now also support SAS/600 and SATA/600, but this has no impact on our performance results at this time, as all flash SSDs are still based on SATA/300.
The SAS HBA MegaRAID 9210-8i is not yet commercially available, but should be released rather soon.
These cards allowed us to reach a maximum read throughput of 3.5 GB/s and an average of 3.4 GB/s on our test system by simply adding all 16 SSDs into one huge RAID 0 array. All I/O performance benchmarks depend a bit on CPU performance, which is why the fully-featured LSI 9260-8i may be quicker in some I/O benchmarks (not in all, though). We tested on the latter card using RAID 0 hardware arrays and a RAID 0 stripe set that we based on the two RAID 0 volumes. This product was limited to 3.0 GB/s throughput.
In the end, it was easy to exceed the former throughput record of 2.3 GB/s, but it wouldn’t have worked without LSI’s support in providing two 9260-8i cards and Intel forwarding the brand new 9210-8i cards. Tweaks such as processor overclocking could certainly further improve performance, but having more bandwidth on the PCI Express side, as well as on the individual storage ports, makes clear that the SSD vendors now have to come up with faster solutions. The architecture is here. Now we're waiting on faster drives.
