802.11ac Wi-Fi Router Testing: Interference And Workloads

IxChariot TCP And UDP Multipair

We saved the best scenario for last. Most homes don’t have exceptional problems with interference or distance, but we are, on the whole, gradually increasing the number of clients connecting to our LANs. All it takes is a couple of kids with tablets, and you’ve got potentially three concurrent media streams plus a couple of guests pulling data to their phones plus whoever is surfing the Web...and so on. The idea of having six or more network clients active in a mainstream home is no longer crazy. Hopefully, at least some of these will be wired, but odds are that at least some will be wireless, and your router has to be up to the job if you want everyone to stay happy.

As you might imagine, running repeatable content to two, four and eight clients simultaneous for all of these tests would be questionably accurate and unduly laborious. Fortunately, we can find a close proxy in IxChariot’s multipair simulations, which also allows us to isolate TCP and UDP traffic.

Keep in mind that the baseline tests use only one pair—data flowing to one client and back from that client. In these tests, we’re simulating multiple clients. If the router can handle the extra load, we should see performance scale up as more pairs get added. For an analogy, imagine you’re trying to figure out how many people can go up a narrow escalator in one minute. If you send them one at a time (one client), you’ll get your baseline. If you gather your crowd into pairs and have them hold hands (two clients), they might get a little crowded standing side-by-side, but overall you’ll get more people to the top in a minute. When you send groups four at a time, you might see more incremental improvement, or you might have your people jostling for space so much that they have trouble getting processed. By the time you reach groups of eight, you better have someone at the escalator helping to organize people as they get on, otherwise you’ll have a mess if not a fistfight.

We’ll provide the baseline again as a reference. Keep in mind that we’re only interested in the average throughput results.

We clearly see Amped’s RTA15 improve as we add clients, though the gains are modest. The difference in total throughput from one client to eight is only 13%, and the needle hardly budges in going from four clients to eight. Asus, sad to say, clearly buckles under even two clients. It tries to recover with four but still doesn’t even come close to its single-client throughput. The same holds true for the R6300. Only the Nighthawk takes a deep breath and keeps lifting as more plates keep going on the barbell. Admittedly, we don’t see much improvement from four to eight, but at least it doesn’t buckle under the load.

If you’re curious about what’s happening under the hood, check this out these eight-pair charts from the Asus (left) and Nighthawk:

In general, we saw routers spend the first 10 to 20 seconds of this test exhibiting very divergent performance across their eight pairs. It was as if the routers were organizing streams and allocating resources on the fly. Once they got things figured out, streams performed in lockstep. But as you can see, lockstep or not, Asus struggles to maintain a steady data flow over the test’s run. In comparison, the Nighthawk’s graph almost looks like a mountain lake at sunrise, smooth as glass and ready for some serious skiing.

When we switch from 5 to 2.4GHz, still sticking with TCP traffic only, we get a different story. Amped and Asus both show no ability to scale beyond one client. The R6300, strangely, finds its legs and doubles its throughput when moving from one to two clients, but then won’t budge any higher. The Nighthawk sees a little lift when stepping up from one client, but then, like the R6300, quickly hits its ceiling. Amped actually has the higher total throughput under load here, but we had to give the win to Netgear since the RTA15 lost 20% of its performance in the shift.

For media streaming, we care most about UDP performance. We limited our testing to two and four pairs because none of the routers could do eight UDP pairs without either IxChariot puking up warnings or our results showing an extraordinary amount of lost data. How much is extraordinary? While there is no official ratio, we figured that anything above 25% constituted so much loss that throughput results would be somewhat meaningless. For example, a two-pair UDP test should, given our parameters, transmit 2000 timing records. If the results only show 1000 records received, then that represents a 50% data loss. You’re not going to get much enjoyment out of half of a data stream.

But again, the Nighthawk is clearly the best bet. All routers do manage to increase UDP performance on 5GHz as we increased client load, but all except the Nighthawk reveal significant error strain with even four clients.

Again, we walk away disappointed with 2.4GHz scaling with UDP traffic. While the Nighthawk remains the only router free from errors and marked data loss, not even that unit could eke out a performance gain beyond one client. We hand Amped the win since it's the only router to show any positive scaling, however slight, although we’d probably still pick the Nighthawk for this application in order to have higher-quality streams.

  • jacobian
    In "How we tested" you didn't specify what kind of wi-fi hardware the test clients had. Was it PCIe or USB network adapter? What kind of MIMO setup, e.g. 3x3 or 2x2?

    Reply
  • chimera201
    I would like to see tear-down of routers to see the components for making a guess at longevity.
    Reply
  • CaedenV
    This was a much better test setup, and a really interesting read. Only complaint was that the nighthawk may have an unfair advantage in the interferance tests compared to the other units as it went up against the 66U instead of another nighthawk. Perhaps you could dig up a 5GHz N router to act as a standard interference device next time? Or find a cheap AC router? It would make it a much better apples-to-apples comparison.

    At any rate, great article! I will certainly send this to my more tech-savvy friends who are looking to upgrade from N right now. I have the 66U in my own home and this seems to be spot-on which what I experience relating to distance and obstructions. It is not necessarily the fastest router on the market, but it is nice an consistent which has merits.
    Reply
  • dgingeri
    Home router security is so poor, I wouldn't even think of using one. Let me guess, they all have WPS, right? HUGE security hole that nobody ha made one little tiny step in fixing, and is "featured" on nearly every home router.
    Reply
  • Larry Litmanen
    I use to have constant WiFi issues, one day i went out and got the best router i could, still had issues. Calling them for support was hell, 45 minutes on the phone just to be told "Sorry sir but there's nothing wrong with our router, your internet is down".

    I found out that my cable/internet company provides a free router, they hooked it up..............no issues since. Heck they just gave me a free AC router that can be controlled from internet.

    I will not be buying any more routers.
    Reply
  • @dgingeri, I've never seen a router that didn't have the option to disable WPS. And WPS is better than nothing for non-technical people.

    Cool article.
    Reply
  • dgingeri
    15852608 said:
    @dgingeri, I've never seen a router that didn't have the option to disable WPS. And WPS is better than nothing for non-technical people.

    Cool article.

    WPS can't be turned disabled by software. That's the big problem with it. My last two routers and one AP got hacked through WPS, even though it was disabled in software. That's when I learned that it doesn't actually get turned off. The only way to protect against the WPS security hole is if the software running the router or AP doesn't support WPS. This can be done by either getting an AP or router without that feature (quite difficult and rather expensive) or putting an opensource software on your router that doesn't have support for WPS (more difficult, but somewhat less expensive.)

    As for "better than nothing for non-technical people", that's pretty much asinine. That's like saying "someone can't operate the key, so we'll leave this side door open."
    Reply
  • dgingeri
    15852434 said:
    I use to have constant WiFi issues, one day i went out and got the best router i could, still had issues. Calling them for support was hell, 45 minutes on the phone just to be told "Sorry sir but there's nothing wrong with our router, your internet is down".

    I found out that my cable/internet company provides a free router, they hooked it up..............no issues since. Heck they just gave me a free AC router that can be controlled from internet.

    I will not be buying any more routers.

    Did you know the ISP free routers have a back door for support purposes. They couldn't support them if they didn't have that back door. The big problem with that is that every support person, and former support person, for that company knows that back door. A vengeful or malicious former employee could easily hack into any customer's router and insert tracking software or "listen in" on the internet traffic, capturing all your passwords. They also have other massive security holes. ISPs patch their router firmware for bugs or security holes even less often than home router manufacturers. They aren't safe.

    Reply
  • liquidpower
    this is the most important part of the Client how could you not included it?
    In "How we tested" you didn't specify what kind of wi-fi hardware the test clients had. Was it PCIe or USB network adapter? What kind of MIMO setup, e.g. 3x3 or 2x2?
    Reply
  • bikeracer4487
    So in the "Performance without Encryption" section, in the very first test you test how long it takes to copy a 2GB folder, with the results measured in seconds. The AC66U wins with the lowest time, and the R6300 having the SLOWEST time...and yet you wrote this: "The exception is the R6300, which takes a 10% to 20% jump. This is a strong enough leap to propel it into first place in our test. Could it be that the R6300 is in fact a far more capable speedster being held back by encryption processing? Or is it just luck and variable ambient test conditions?"
    Reply