Test Setup And Methodology
For powerline performance tests, there are two key areas of focus: maximum throughput and maximum throughput at varying distances. Because real-world conditions rarely match published numbers, it's important to establish a baseline of how fast the test machines can transmit and receive data with minimal overhead in between. To that end, inserting a switch between them and running a benchmark can give you a good sense of baseline maximum throughput without adding the additional latency of sending packets through a different medium and over larger distances.
Using the same devices, adding in the powerline adapters requires using different outlets. To minimize distance for the initial test, it would be ideal to perform a "same room" test. From there, one powerline adapter can remain stationary while the other one is moved farther and farther away. During each move, a repeat test can be performed for a comparison point to the initial baselines.
All testing was conducted in my house, a 1848 square-foot (172 squar-meter), three-story townhome in a small suburban community in Laurel, Maryland. I decided to test devices in multiple physical locations to see how the powerline adapters performed from varying physical distances, including the second floor to the basement, the second floor to the first floor, the first floor to the basement, within the same floor and within the same room.
Tests were conducted using two systems: a "server" laptop and a "client" laptop, dubbed Laptop A and Laptop B, respectively. For the majority of the tests, Laptop A remained on the first floor of my townhome, while Laptop B was moved to the varying locations. Each powerline adapter reported a 1 Gb/s connection speed during testing. Prior to measuring throughput using the Powerline adapters, I plugged both laptops into the switch for a direct laptop-to-laptop baseline in an effort to establish a more realistic (and potential) maximum performance ceiling. According to that test, the maximum TCP throughput achieved was around 700 Mb/s, with the maximum UDP throughput reaching around 300 Mb/s.
Throughput was tested using a laptop-to-laptop 2GB folder transfer using Microsoft's native file sharing, as well as a laptop-to-laptop data transfer using PassMark's PerformanceTest 8 advanced networking test. PassMark provides a throughput tester that allows the reviewer to select the mode, Internet Protocol version, transfer protocol, and data parameters. Each test ran for three minutes before halting, except in certain cases where I could tell the UDP throughput was flooding the switch. An easy way to tell that this is happening during your tests is to try to connect to the end point on the other side. If you can't, all signs point to a switch flood.
Copying the 2GB folder was as easy as sharing a folder on each laptop and dumping in enough files to reach the 2GB mark. I would've used the venerable JPerf to monitor this 2GB folder transfer, but my five seconds glancing at the SourceForge and Google code pages made me think I would have to run it with Linux. Turns out, it works in Windows. In future testing, JPerf will be added in the list of software tools that test throughput in Windows.
In researching other Windows-based throughput monitoring tools, I came across NetWorx. It's similar to JPerf, but I haven't done a complete comparison to cross-check features and capabilities. The only ding against testing with NetWorx is that you have to start and stop its graphing and reporting before and after the test, so I couldn't run a test and then go back later to pull the report for a specific test. I guess the real feature missing when compared to PassMark would be a testing option that completed on its own and saved the results in its own data set.
The software was running on each laptop used in the test, and metrics were recorded at each end when my benchmark finished. One thing to note is that, in Windows 7 and up, Microsoft provides emulated network drivers to ensure the functionality is present, but that doesn't mean that the emulated driver provides the same throughput! During the initial phase of my testing, my 2GB folder transfer would die out occasionally, and I'd have to repeat the test multiple times until I achieved success. Since I needed to run multiple tests with each set of powerline networking adapters, I decided to troubleshoot the issue. It turns out that, after I installed the manufacturer's network adapter driver, my sessions no longer died out! I was worried that it might also affect my maximum achievable throughput, so I retested.
Test Systems
| Header Cell - Column 0 | Test Laptop A "Client" Specs | Test Laptop B "Server" Specs |
|---|---|---|
| Model | Sony VAIO VPCF226FM | Gateway NV59 |
| Processor | Intel Core i7-2630QM (Sandy Bridge) @ 2GHz, 6MB Shared L3 Cache, Four Cores, Hyper-Threading Enabled | Intel Core i3 (Arrandale) @ 2.13GHz, Two Cores, Hyper-Threading Enabled |
| Memory | 8GB Hynix (2 x 4GB) DDR3 SDRAM, PC3-10600 | Elpida 4GB (2 x 2GB) DDR3-1066, PC3-8500S-7-10-F1 |
| Storage | OCZ Agility 3 AGT3-25SAT3-240G 240GB SSD | Samsung 830 Series 128GB SSD |
| Graphics | Nvidia GeForce GT 540M | Integrated Intel HD Graphics |
| Power Supply | Sony VGP-AC19V54 | Delta Electronics ADP-65JH DB |
| Network | Realtek PCIe Gigabit Ethernet Family Controller | Broadcom NetLink Gigabit Ethernet |
| Operating System | Microsoft Windows 8.1 Professional (64-bit) | Microsoft Windows 8.1 Professional (64-bit) |
Benchmark Suite
| Networx | Free tool that enables objective review of bandwidth consumption either cumulatively or per interface. |
|---|---|
| Incoming and outgoing traffic is represented on a line chart and logged into a file for later review. In addition, reports can be exported to a variety of formats, such as HTML, MS Word and Excel, for further analysis. | |
| Networx Test Conditions | 2 GB folder copy using native Microsoft File Shares at varying distances: same room, same floor, different floors |
| Passmark Performance Test 8 | Designed to test the data transfer rate between two computers both of which must be running PerformanceTest. One of the computers must act as the server and will sit waiting for a connection. The other computer acts as a client. It connects to the server machine and sends data to it for the duration of the test. |
| Passmark Performance Test 8: Advanced Networking Test Conditions | Three-minute TCP data transfer over IPv4 using a Fixed Block size of 16,384 bytes |
Although the suggested PassMark Performance Test benchmarks used the default settings, additional testing could have been performed using fixed blocks of smaller and larger sizes and/or using variable block sizes. Because there were so many distances tested, adding in any changes from the default test settings would inflate the test results to a more unwieldy length. Some people may point out that fixed block sizes may not be an accurate representation of the varying data that would actually flow through a powerline adapter if it were intended to be used for Internet access and/or streaming media delivery. Please add your comments below if you'd like us to incorporate variable block sizes into our future test methodology.
MORE: Powerline Networking 101
MORE: Networking in the Forums
Matthew Matchen is an Associate Contributing Writer for Tom's Hardware. Follow him on Twitter @matchemm
Follow us on Twitter @tomshardware, on Facebook and on Google+.
