Disappoint & Strange hardware RAID 5 performance

caveman1983

Reputable
Aug 5, 2014
2
0
4,510
Ladies and Gentlemen:

I have a very strange situation that I hope to get your guys opinion on to see why my RAID array performance is so bad:

Common Settings:
================

* HP Z420 Workstation with E5-1620 Xeon CPU
* Win 7 x64
* LSI 9361-8i RAID Controller with 3rd Gen SAS 12 Gb/s
- Host Bus Type is x8 lane PCIe 3.0
- Internal Ports: 8
- Up to 12Gb/s per port
- 2x SFF-8643 Mini-SAS HD port
* RAID 5 with Megaraid
* Controllers' firmware upated
* MegaRAID setup parameter
- always write back
- always read ahead
- Direct IO
- Disk Cache Policy Enabled

Setup 1: Normal
================================
- 4 WD Black FZEX
- 2 Hitachi Deskstar 7k4000
- Directly link these 6 drive to LSI RAID controller using (2) 1-meter Mini-SAS HD SFF8643 to SATA fanout cables (CBLSFF8643-SATASB-10M)
- 4 drives on one of the cable (CBLSFF8643-SATASB-10M)
- 2 drives on one of the other cable (CBLSFF8643-SATASB-10M)

Setup 2: Suboptimal
==================================
- 4 WD Black FZEX
- 2 Hitachi Deskstar 7k4000
- 3 Hitachi NAS Deskstar 7k4000 (new addition, which is RAID-supported)
- Intel RES2SV240 SAS Expander Card (6Gb/s)
- Link LSI controller with Intel Expander with "SFF-8643 to SFF-8087" (Adaptec 2279700-R ACK-I-HDmSAS-mSAS-1M Cable)
- Link the above 9 hard drive to Intel Expander with three SFF-8087
- 1x 3ware CBL-SFF8087OCF-06M
- 2x areca CB-87SA-75M SFF-8087 MiniSAS to 4 SATA 0.75 Meter Breakout Cable - OEM)
- 4 drives on CBL-SFF8087OCF-06M
- 4 drives on CB-87SA-75M SFF-8087
- 1 drive on the other CB-87SA-75M SFF-8087
- Started CacheCade trial but disabled afterwards (also in the tests below)

Performance Comparision:
========================

Test with CrystalDiskMark

* Setup 1: Normal
---------------------------------
- Sequential Read: 815 MB/s
- Sequential Write: 814 MB/s


* Setup 2: Sub-optimal
---------------------------------
- Sequential Read: 416 MB/s
- Sequential Write: ~ 600 MB/swww.intel.com/go/RAID

Endnote
=========

In my rig, I also have a hardware RAID 0 setup using three Crucial M550 SSD. These drives are linked to the LSI controller with CBLSFF8643-SATASB-10M in both situations described above. Performance is as expected per specification: 1500 MB/sec.
 
In sceanrio 1, are you hooking all 6 drives to 1 SAS port? You don't specific how many drives go to each port.

If they are all on 1 port, each port is capable of 12Gb/s, which is roughly 1GB/s, which is just above your 815MB/s, minus overhead, right on track. If they are all on 1 connection, you are fully using the available 12Gb.
 

caveman1983

Reputable
Aug 5, 2014
2
0
4,510
Hi, getochkn:

I just updated the details. Thank you for your help here. I'm very puzzled.