SMB Performance Testing
Testing Notes
All six of the systems work with traditional SMB/CIFS file transfers. SMB is the easiest NAS protocol because it simplifies file access to where you can simply type in "\\name_of_nas\folder_name" in file explorer to access the data on the system. We test and highlight the performance of SMB encrypted folders and iSCSI volumes on applicable models, but not all of the systems support the features.
Our test system is a modified Quanta MESOS CB220 Cluster-In-A-Box multi-node server. We upgraded the system with two Intel E5-2670 V1 processors and 128GB of system memory per node. An Intel X520 network interface card with two 10GbE ports resides in each node, as well. The system allows us to deploy four test nodes that execute custom software scripts in industry standard test software to measure our base performance metrics.
The test system and devices under test connect via a Netgear M6100 fully-modular switch, which provides ample 10GbE and 1GbE ports to sustain our large testing environment. This round of NAS testing is the second time we've used this switch. We discovered an issue with our previous switch and some NAS products that significantly reduced file transfer performance. The Netgear M6100 doesn't give us the same performance issues, and it allows us to expand our test environment while still running a dedicated network.
Sequential Read Performance
The sequential read tests reveal two distinct performance tiers. You would expect this segmentation with the different processors in these systems, but the dividing line is not that clear.
For this class of product, we are looking for high performance at low queue depths. Most users will use these systems for file transfers in single-user environments, so most of the transfers will take place one at a time (sequentially). Typically, when you copy and paste files from your desktop into a NAS folder, it occurs at a queue depth of one. The queue depth increases to 2 if you start two separate transfers simultaneously, and so on.
We expected the Intel-based systems to dominate nearly all of the tests, but that isn't the case. The Asustor AS3102T, Synology DS216j, and Western Digital My Cloud Mirror provide the best performance at low queue depths with 128BK data blocks. Of the three, only the AS3102T uses an Intel processor.
This behavior highlights that firmware and software play just as large of a role in performance as the hardware does, particularly during light workloads.
Sequential Write Performance
The manufacturers release new firmware at a fairly consistent rate to update the products, increase performance, add features, and smooth performance inconsistencies. Many of the systems use DRAM as a cache buffer, which helps to smooth performance, because hard disk drives, by nature, are inconsistent.
The sequential write tests expose some performance abnormalities. All of the products, except the QNAP TAS-268, provide similar performance while writing 128KB sequential data. At higher queue depths, the TAS-268 matches the performance of the other products, but it lags behind the other NAS during light workloads.
Sequential Mixed Workload Performance
The sequential sweeps test cycles through mixed workloads in 10-percent increments. We begin with 100-percent reads (on the right of the chart) and add more right data until we reach the left side of the chart with 100-percent writes. We highlight the 70-percent read test with additional charts because this is the standard mixture for measuring workstation storage performance.
The Western Digital My Cloud Mirror performs very well in these tests while the other products follow the traditional "bathtub curve."
Random Read Performance
The Thecus N2810 performs slightly better with random read data across the range of queue depths. The Synology DS216+ trails the other products in these SMB tests. We spent a considerable amount of time retesting the DS216+ in an attempt to smooth out the issue. Each system requires eight days to complete the full test suite, and we retested the DS216+ with four different software releases, but could not isolate the issue.
The other NAS appliances converge together and provide nearly identical random read performance.
Random Write Performance
The random write test spreads the test pool out and gives us more separation. Most of the products are very consistent and waver only slightly as the queue depth increases, but the Synology DS216+ is, for lack of a better term, all over the place. This pattern reveals high performance as the cache algorithm buffers the incoming data and then low performance as the NAS flushes the buffer to the drives.
Random Mixed Performance
The mixed random test shows how the cache allows the systems to absorb the incoming data at a higher rate than they can read it back to the host system. Again, we highlight the 70-percent read workload with additional charts to characterize workstation performance. 4KB transactions simulate running Windows-based software over the network, and 8KB data highlights VMware performance trends.
Standard Server Workloads
Very few users, in any, will run a heavy-use email or database server on these entry-level systems, but we like to see the tests to examine the mixed-workload performance in the latency over IOPS charts.
Consumer NAS Workloads
Normally we sort the different products by arranging them based upon performance (best on top, worst on bottom). However, that makes isolating a specific model difficult in this case, because every model on the list is under examination.
Intel's NASPT program tests with recorded traces, which highlights real-world performance better than other techniques. We use the longer batch test setting to record the results, which repeats the workload three times and then provides us with the average results.
Most of the systems performed well in some tests but suffered in others. The Asustor AS3102T and Thecus N2810 provided relatively high performance during each workload, and they also stand out for delivering balanced performance.
