802.11ac Wi-Fi Router Testing: Interference And Workloads

Baseline 5GHz Tests

Most of our tests were run in a living room, with the main system and router at one end of the room and the client #1 machine in the other. We had line of sight connectivity between the devices over a span of about 30 feet. All of these initial baseline tests were run with the routers applying WPA2 encryption, as this is the most common setting for home router deployments.

Our 2GB test involves taking several hundred small files (MP3s, documents, etc.) and compressing them into a single ZIP. This helps to isolate our results from the added processing overhead of sending hundreds of files rather than one.

There’s nothing fancy about this test. It’s just a straight up file transfer to establish a baseline for comparison. Under fairly ideal conditions, our Asus router surprisingly wins the event, although the Netgear R6300 shows the narrowest performance gap between the two transfer directions.

PassMark’s PerformanceTest 8 offers another look at basic throughput, only now we can assess performance of TCP and UDP traffic separately. UDP traffic almost invariably exhibits higher throughput than TCP since it doesn’t involve the latter’s error-checking (and thus packet resending) routines. If a packet fails to transfer, UDP keeps right on going. Here’s a side-by-side look at the potentially extreme difference between TCP (left) and UDP traffic as seen on our Asus router, albeit in a different environment before we started official testing. Same distance, same test, just different packet traffic protocols.

Now back to our comparative PassMark results:

Since we see four different instances of throughput hitting in the 410 to 413 Mb/s range, it seems clear that we’ve reached the ceiling for performance given this benchmark’s parameters. Note, however, that while three of our four routers hit this level for UDP traffic, only the Nighthawk R7000 reach it with TCP traffic. Amped’s RTA15 is the only router not to hit this ceiling with UDP, although it comes close.

Now let’s see if the more in-depth IxChariot corroborates our earlier tests. Remember when we talked about biasing for average throughput first and then falling back on lowest throughput to decide a close race? This is one of those times. All three of our 3x3 routers show similar average and top-end speeds during 5GHz TCP testing. Asus, however, wrecks both Netgear models in maintaining a better throughput floor. Amped also fares well in this regard, leading us to think that all of that power boosting might be making some amends for only having a 2x2 antenna array.

When we performed testing on the Amped RTA15, IxChariot tossed us warning message CHR0336: A timing record was received with a measured time between 1 and 20ms. While the other three routers go quietly about their business (with the Nighthawk showing only the barest advantage), Amped is all over the map, like so:

Compare this to the much more tightly fluctuating Nighthawk on the same test:

We’ll see more of this in the tests to come. Amped struggles with UDP in general, especially across 5GHz.

A final note here. You will notice that our 5GHz TCP results are significantly faster than our UDP results. This is generally not how it works in the real world, as is reflected in PerformanceTest. However, as we discussed in our prior 11ac round-up, Ixia appears to use a custom protocol implementation that skews the relative performance of TCP and UDP.

  • jacobian
    In "How we tested" you didn't specify what kind of wi-fi hardware the test clients had. Was it PCIe or USB network adapter? What kind of MIMO setup, e.g. 3x3 or 2x2?

    Reply
  • chimera201
    I would like to see tear-down of routers to see the components for making a guess at longevity.
    Reply
  • CaedenV
    This was a much better test setup, and a really interesting read. Only complaint was that the nighthawk may have an unfair advantage in the interferance tests compared to the other units as it went up against the 66U instead of another nighthawk. Perhaps you could dig up a 5GHz N router to act as a standard interference device next time? Or find a cheap AC router? It would make it a much better apples-to-apples comparison.

    At any rate, great article! I will certainly send this to my more tech-savvy friends who are looking to upgrade from N right now. I have the 66U in my own home and this seems to be spot-on which what I experience relating to distance and obstructions. It is not necessarily the fastest router on the market, but it is nice an consistent which has merits.
    Reply
  • dgingeri
    Home router security is so poor, I wouldn't even think of using one. Let me guess, they all have WPS, right? HUGE security hole that nobody ha made one little tiny step in fixing, and is "featured" on nearly every home router.
    Reply
  • Larry Litmanen
    I use to have constant WiFi issues, one day i went out and got the best router i could, still had issues. Calling them for support was hell, 45 minutes on the phone just to be told "Sorry sir but there's nothing wrong with our router, your internet is down".

    I found out that my cable/internet company provides a free router, they hooked it up..............no issues since. Heck they just gave me a free AC router that can be controlled from internet.

    I will not be buying any more routers.
    Reply
  • @dgingeri, I've never seen a router that didn't have the option to disable WPS. And WPS is better than nothing for non-technical people.

    Cool article.
    Reply
  • dgingeri
    15852608 said:
    @dgingeri, I've never seen a router that didn't have the option to disable WPS. And WPS is better than nothing for non-technical people.

    Cool article.

    WPS can't be turned disabled by software. That's the big problem with it. My last two routers and one AP got hacked through WPS, even though it was disabled in software. That's when I learned that it doesn't actually get turned off. The only way to protect against the WPS security hole is if the software running the router or AP doesn't support WPS. This can be done by either getting an AP or router without that feature (quite difficult and rather expensive) or putting an opensource software on your router that doesn't have support for WPS (more difficult, but somewhat less expensive.)

    As for "better than nothing for non-technical people", that's pretty much asinine. That's like saying "someone can't operate the key, so we'll leave this side door open."
    Reply
  • dgingeri
    15852434 said:
    I use to have constant WiFi issues, one day i went out and got the best router i could, still had issues. Calling them for support was hell, 45 minutes on the phone just to be told "Sorry sir but there's nothing wrong with our router, your internet is down".

    I found out that my cable/internet company provides a free router, they hooked it up..............no issues since. Heck they just gave me a free AC router that can be controlled from internet.

    I will not be buying any more routers.

    Did you know the ISP free routers have a back door for support purposes. They couldn't support them if they didn't have that back door. The big problem with that is that every support person, and former support person, for that company knows that back door. A vengeful or malicious former employee could easily hack into any customer's router and insert tracking software or "listen in" on the internet traffic, capturing all your passwords. They also have other massive security holes. ISPs patch their router firmware for bugs or security holes even less often than home router manufacturers. They aren't safe.

    Reply
  • liquidpower
    this is the most important part of the Client how could you not included it?
    In "How we tested" you didn't specify what kind of wi-fi hardware the test clients had. Was it PCIe or USB network adapter? What kind of MIMO setup, e.g. 3x3 or 2x2?
    Reply
  • bikeracer4487
    So in the "Performance without Encryption" section, in the very first test you test how long it takes to copy a 2GB folder, with the results measured in seconds. The AC66U wins with the lowest time, and the R6300 having the SLOWEST time...and yet you wrote this: "The exception is the R6300, which takes a 10% to 20% jump. This is a strong enough leap to propel it into first place in our test. Could it be that the R6300 is in fact a far more capable speedster being held back by encryption processing? Or is it just luck and variable ambient test conditions?"
    Reply