How We Test Network Switches

Introduction

Despite all of the mobile hardware that wireless connections service, the fastest wireless technologies can't compete with the fastest wired networks. A good Ethernet switch serves up exceptional performance, while enabling the features and capabilities needed to build robust home, small business and enterprise networks. If you would like to know more about how switches work, be sure to read our Network Switch 101.

Networking switches take many forms, ranging from managed and unmanaged switches to those armed with Power over Ethernet and other advanced features. There are many ways to compare and contrast networking hardware, but the Tom's Hardware team created a suite to reflect what we felt were the most important metrics: throughput and response time. Our methodology involves multiple benchmarks, some of which illustrate peak performance, and some designed to generate real-world results under plausible work conditions.




Test System Specifications

We use four systems for our network switch testing:

Test Bench: Tom's Hardware Reference System

Loading...

Test NUC: Intel NUC5i7RYH

Test Server: ASRock Vision X 471D

Test Laptop: Sony VAIO SVS13112FXS

MORE: All Networking Content
MORE: Network Switch 101

Testing Suite And Methodology

Our testing suite is split into four different benchmarks: Point to Point, Bi-Directional, Mesh Interference and Response Time. Tests are conducted using Ixia's IxChariot software to measure throughput and response time between systems.

Before the switches are tested, an Ixia endpoint agent is installed on our three client systems (the server is already running IxChariot and does not need another endpoint). Downloadable from the Ixia website, the agent uses a preconfigured script to measure and report the results we're looking to generate via IxChariot's interface.

A control is a test in which the subject is not influenced by variables. In this case, our control is a straight cable test; no Ethernet switch is involved. Throughput is measured from point to point between the ASRock VisionX and Sony Vaio.

Using IxChariot and each computer' IP address, we created an endpoint pair, designating our ASRock Vision X server as Endpoint 1 and our Sony VAIO as Endpoint 2. We chose a script aimed at measuring throughput over Gigabit Ethernet; the same script was used for our Point to Point, Bi-Directional and Interference tests. All other settings remained the same.

Under Run Options, "Run for a fixed duration" is set to one minute. The control test and all subsequent tests are conducted under a one-minute cap.

Once the test is complete, the Throughput tab illustrates the minimum, maximum and average speeds. The line graph below illustrates performance at each point in time. We'll be comparing this data to the results from our Point to Point, Bi-Directional and Mesh Interference tests later.

Our first variable metric is the Point-to-Point TCP Throughput Test, which measures the throughput from one endpoint to another. The process is exactly the same as the Control Test, but it runs through our network switches. As in the Control Test, the active members are the ASRock VisionX and Sony VAIO laptop; the other two computers are offline.

The image above shows how IxChariot charts its results. In this Point-to-Point Throughput Test, you can see a slight but evident difference between the Control Test and Network Switch runs. The delta isn't particularly noteworthy; after all, adding a switch simply introduces a bridge for information from the server to travel through—a lone traveler on an empty bridge should not encounter much resistance on its journey. Likewise, our stream of benchmark data is not obstructed.

Testing throughput from the server to the laptop (and vice versa) over a network switch introduces more network traffic and produces lower average throughput. As with the previous benchmark, one pair designates the server as Endpoint 1 and the laptop as Endpoint 2. A second pair is created in IxChariot with the laptop as Endpoint 1 and the ASRock VisionX as Endpoint 2.

This chart shows the effect of driving more traffic through a network switch, demonstrating lower throughput across the board. In our bridge allegory, the traffic would be two travelers crossing the same bridge. Depending on how wide and well-built the bridge is, they might bump into each other, brush shoulders or walk past each other effortlessly.

The Mesh/Interference Test generates even more traffic through the switch. The results from the Mesh Test will suggest the performance level you might expect in a practical setting. After all, network switches are meant to have multiple devices connected to them. We want to stress each switch by flooding it with increasing amounts of data. Adding traffic in measured increments gives us an idea of how well the switch performs with two, three, four or more clients attached.

This test splits connections into three pairs: test bench to the NUC, NUC to the server, and server to the laptop.

In this example, more travelers with different destinations use the bridge, and contact with other travelers slows them down. Adding more connections to the switch diminishes overall throughput. Certain pairings, such as that of our Intel NUC to the server, delivered lower average, minimum and maximum throughput in multiple tests. If our bridge were bigger and sturdier, there would be less interference. A business-class switch, for example, is meant to handle high levels of traffic with numerous systems connected.

Response Time Tests are a little bit different. The same procedure for creating a Point-to-Point Test is used—a pairing between the ASRock VisionX and Sony VAIO laptop is created, and the run duration is set to one minute. But we use a different script tailored to measuring the response time of the switch.

After the test is completed, this error message should appear: "IxChariot cannot show the response time if the results are below 20 milliseconds." To find the results of the test, switch to the Transaction Rate tab rather than the Response Time tab. There, you find the Transaction Rate average. To find the response time, use the following equation: 1/Average Transaction Rate x 1000 = Response Time

The lower the number, the faster the response time.

In this sample, the Transaction Rate average is 3437.435. Divide 1 by 3437.435, and multiply the result by 1000. The response time is rounded to 0.291 milliseconds.

Response times slightly from switch to switch. And at under one millisecond, you won't notice a difference from one model to another. But knowing exactly how fast switches perform may help you get the best value for your dollar.

Final Thoughts

As mentioned in the introduction, there is lots of variety in the network switch market. Each manufacturer sells different models armed with distinct feature sets. When we first started testing Ethernet switches, we devised a test suite that catered to as many products as possible, knowing that we'd have to be adaptable—so consider this How We Test a version 1.0.

In the coming weeks, we will consider different approaches for testing specialty switches, such as managed switches and smart switches. Any suggestions regarding revisions to our current test suite and/or new procedures for specific products are always welcome!

MORE: All Networking Content
MORE: Network Switch 101

Alexander Quejado is a Contributing Lab Assistant for Tom's Hardware. Follow him on Twitter and Facebook.

Follow us on FacebookGoogle+RSSTwitter and YouTube.

Create a new thread in the US Reviews comments forum about this subject
This thread is closed for comments
15 comments
Comment from the forums
    Your comment
  • allenpan
    quick question what testing software is it?
  • zodiacfml
    Nice but can we get something interesting or controversial when reviewing network switches? Increase the difficulty. Challenge yourselves; review various Wi-Fi routers especially the MU-MIMO capable.
  • Rancifer7
    I like it, can't wait to see this put to good use on some network switches with mixed wireless too.
  • blackmagnum
    While you're busy with this, please consider testing the different brands of lan cables as well. I've heard that the monster cables are the market benchmark.
  • jacklongley
    Anonymous said:
    quick question what testing software is it?


    It's IxChariot by Ixia (formerly by NetIQ). Frankly, not the best of testing solutions, but beggars can't be choosers.
  • jacklongley
    Anonymous said:
    While you're busy with this, please consider testing the different brands of lan cables as well. I've heard that the monster cables are the market benchmark.


    None of the tests presented will demostrate a difference with network cabling, insomuch as any cabling certified at Cat5E or above will allow the devices to operate at 1Gbps (or lower, for those devices using FastE instead).

    Cabling benchmarks are more around the available transmission headroom, i.e. which Ethernet standard will it support. You can already ascertain these from the category of cabling, e.g. Cat7 vs. Cat5.
  • jacklongley
    Personally, I'm not impressed with the lackluster approach to testing that Tom's Hardware demonstrates for network devices. These types of tests should follow the testing methodologies laid out years ago in RFC2544. In fact, IxChariot should already have RFC2544 options available by default.

    Perhaps Tom's should leave network testing and articles to the big boys.
  • VTOLfreak
    I'm more upset about the hardware they are using than the software. You'd expect someone running network tests to at least bring server class network cards to the table. What are you testing here guys? The switch in question or your crappy network cards?
  • irongnome
    I have a few metrics that I would be very interested in seeing measured.
    A. Does the switch have adequate buffers/properly implemented flow control. This comes in to play when you mix FastEthernet and Gigabit on the same network.
    B. Full Line Rate and Latency Test using a snake topology. Eg. Ports 1 and 2 on VLAN 1, Ports 3 and 4 on VLAN 2, Ports 5 and 6 on VLAN 3, etc. With a patch cable bridge 2 and 3, 4 and 5, etc. Connect the test machines to the first and last port. This will load the switch to its absolute capacity while only requiring two endpoints.
    C. Power consumption.
  • DrakeFS
    Quote:
    While you're busy with this, please consider testing the different brands of lan cables as well. I've heard that the monster cables are the market benchmark.


    I truly hope you are being sarcastic, as networking is a digital signal. As a previous poster stated, if the cable is rated at 5e or above, it will serve up to 1Gbs of bandwidth. 6 and 6a will serve up to 10Gbs of bandwidth. The only thing that a Monster cable may have better than your average 5e ethernet cable, is that they may be terminated within tighter specifications. Still, this will not affect most consumers as the bottle neck will be your ISP speeds(unless you are one of the lucky ones with >= 1Gbs service). If you need to do a lot of cabling (or some really long cables), I suggest purchasing a 1000ft roll of 5e or 6a, some RJ45 terminators and a crimping tool. You will save $$$ in the end.
  • skit75
    Monster would have you believe their cables can stop DDOS attacks. That is why you pay $50.00 for a 6' CAT5e cable.... >=)
  • THJulio
    Quote:
    Nice but can we get something interesting or controversial when reviewing network switches? Increase the difficulty. Challenge yourselves; review various Wi-Fi routers especially the MU-MIMO capable.


    Thanks. We're currently building out a separate router category slated to start up in Q1 2016.
  • THJulio
    Quote:
    I'm more upset about the hardware they are using than the software. You'd expect someone running network tests to at least bring server class network cards to the table. What are you testing here guys? The switch in question or your crappy network cards?


    Quote:
    Personally, I'm not impressed with the lackluster approach to testing that Tom's Hardware demonstrates for network devices. These types of tests should follow the testing methodologies laid out years ago in RFC2544. In fact, IxChariot should already have RFC2544 options available by default.

    Perhaps Tom's should leave network testing and articles to the big boys.


    Thanks for the input. We're still building out our test suites for the different networking devices, and as the article mentions, these current benchmarks are basically at v1.0. In their current state, we want the tests to cover real world conditions. In the next version, we'll be adding in stress test scenarios using server/workstation level hardware that will push the switches and routers even more.
  • dE_logics
    I call it Windows switch test rig.

    All because of the limitations of your OS and Windows 'software', you can't use a single PC with 4 network cards/ports to do the same tests.

    Apart form that, it appears you test your switches for 'windows readyness'.
  • VTOLfreak
    Quote:
    I call it Windows switch test rig.

    All because of the limitations of your OS and Windows 'software', you can't use a single PC with 4 network cards/ports to do the same tests.
    Windows as a testing platform is not a problem but you need the right combination of hardware, drivers, applications and so on to do proper offloading. You need to be allot more careful on Windows than on say Linux or BSD. You miss one step somewhere and the whole thing falls over and you are standing around wondering why you are only getting a few gbit out of a 10gbit port on a windows file server for example. Irony is that once you do all that you have pretty much bypassed the entire network stack on Windows so in that respect I do have to agree with you. :)