How We Tested
Again taking a chapter out of our beamforming study, we went into this comparative study with an eye toward UDP performance. Ultimately, blistering TCP performance is great for large file transfers, but if it takes an extra minute or two to transfer 10GB into network storage, no one is going to lose any sleep over it. Moreover, with TCP, you know that data is going to arrive at its destination intact because the protocol mandates confirming that packets are received error-free at their proper destination. This is great for data integrity, not so great for overall throughput.
UDP makes no such demands on data. With UDP, packets simply blast from a source computer to their target, and if they get lost along the way, well, them’s the breaks. If a relay node between the source and target happens to be congested, the stream can come to a grinding halt. Or packets might arrive out of order (TCP also mandates preserving proper packet order). Admittedly, UDP transfers usually come off flawlessly, and stipping away all of that error-checking and receipt confirmation often helps the protocol run circles around TCP. This is why video and network gaming apps increasingly gravitate to UDP. But UDP is simply more prone to losing packets, in rare cases exhibiting up to 5% data loss.
Fortunately, these are extreme cases. In fact, we omitted packet loss from the following charts because there were none. That’s right. Across all tested devices, both with UDP and TCP, we didn’t register a single dropped packet. Obviously, this is great news and should help give confidence to anyone wondering about data integrity with these different connection technologies. Whatever variables exist within a common home, apparently they aren’t severe enough to impair packet transmission at these relatively close ranges.
We tested with two applications, the commonly referenced Iperf and the Zap utility from Ruckus Wireless, detailed in our beamforming coverage. Both programs are able to test TCP and UDP transfers. For Iperf, we used a server command of Iperf.exe -s -w 10000k and a host command of Iperf.exe -c 192.168.1.3 -P 2 -i 1 -p 5001 -f k -t 10. Note that the following image was taken using Iperf’s GUI-based brother, Jperf.
We’re particularly fond of Zap because of its ability not only to provide average throughput rates but, more importantly, minimum transfer rates. As we wrote in the beamforming article: “[A Ruckus benchmarking] transfer is divided into percentages of the total workload, with each step being one-tenth of a percent. At each step, throughput rate is recorded and the number shown by the software is the lowest packet speed recorded up to that point in the transfer job. This is why Zap numbers look really fast at 1%, average at 50%, and very slow at 99 percent. For our purposes, we’re most interested in the average and lowest numbers. When it comes to video, you don’t care what the fastest or average sustained rates are. You care about the slowest speeds, the weakest link in the wireless chain, because this will be the key factor in determining your video watching [or gaming] experience. If you sustain a 70 Mb/s connection 95% of the time but occasionally drop to 15 Mb/s for whatever reason, then those drops are going to translate into dropped frames and hiccups if you’re watching an HD stream with a 19.2 Mb/s data rate.”
We should note that we shared our test results with Netgear prior to publication. There were only a few points where the company took issue with our results and methods. One of these was the inclusion of Gigabit Ethernet numbers as a reference point being held up against the other three technologies. As one product manager commented, “Fast Ethernet [is] a more typical comparison. If a person has Gigabit Ethernet in their home, they obviously wouldn't be considering any wireless, powerline, or MoCA solutions. But MoCA, as an example, might actually be a viable alternative to 10/100.” This is a valid point, but ultimately we decided to stick with Gigabit Ethernet results because with each passing month the technology is more commonplace, even in the mainstream. Few people would endure the expenses in time or money to string Fast Ethernet in a structured wiring retrofit, but Gigabit Ethernet is another story. Gigabit is a viable upgrade option, and as such it should be weighed alongside these other three alternatives.
