Sign in with
Sign up | Sign in
Your question
Closed

Gigabit Ethernet: Dude, Where's My Bandwidth?

Last response: in Reviews comments
Share
June 22, 2009 6:28:35 AM

why is the RAM-to-RAM network max speed on the graph 111.2 when u state 111.4? typo?
June 22, 2009 6:29:41 AM

Interesting article, thank you. I wonder how a hardware based RAID 5 would perform on a gigabit network compared to a RAID 1?
Related resources
Anonymous
June 22, 2009 6:55:36 AM

Hello

Thanks for the article. But I would like to ask how is the transfer speed measured. If it is just the (size of the file)/(a time needed for a tranfer) you are probably comsuming all the bandwith, beacuse you have to count in all the control part of the data packet (ethernet header, IP headrer, TCP header...)

Blake
June 22, 2009 7:13:37 AM

The article does not make any sense and created from an rookie. Remember you will not see a big difference when transfer small amount of data due to some transfer negotiating between network. Try to transfer some 8GB file or folder across, you then see the difference. The same concept like you are trying to race between a honda civic and a ferrari just in a distance of 20 feet away.

Hope this is cleared out.
June 22, 2009 7:14:41 AM

Don Woligroski has some incorrect information, which invalidates this whole article. He should be writing about hard drives and mainboard bus information transfers. This article is entirely misleading.

For example: "Cat 5e cables are only certified for 100 ft. lengths"
This is incorrect. 100 meters (or 328 feet) maximum official segment length.

Did I miss the section on MTU and data frame sizes. Segment? Jumbo frames? 1500 vs. 9000 for consumer devices? Fragmentation? TIA/EIA? These words and terms should have occurred in this article, but were omitted.

Worthless writing. THG *used* to be better than this.
June 22, 2009 7:23:21 AM

Quote:
There is a common misconception out there that gigabit networks require Category 5e class cable, but actually, even the older Cat 5 cable is gigabit-capable.


Really? I thought Cat 5 wasn't gigabit capable? In fact cat 6 was the only way to go gigabit.
June 22, 2009 7:25:46 AM

why didn't you test SSD performance? It's quite a hot topic and I'm sure a lot of people would like to know if it will in fact improve network performance. I can venture a guess but it'll be entirely theoretical.
June 22, 2009 7:34:15 AM

Gbit is actually 10^9 bits per second, ie about 119 MB/s.
June 22, 2009 7:37:04 AM

do you have any engineers on your staff that understand how this stuff works?? when you transfer some bits of data over a network, you don't just shoot the bits directly, they are sent in something called packets. Each packet contains control bits as overhead, which count toward the 125 Mbps limit, but don't count as data bits.

11% loss due to negotiation and overhead on a network link is about ballpark for a home test.
June 22, 2009 7:41:35 AM

After carefully read the article. I believe this is not a tech review, just a concern from a newbie because he does not understand much about all external factor of data transfer. All his simple thought is 1000 is ten time of 100 Mbs and expect have to be 10 time faster.

Anyway, many difference factors will affect the transfer speed. The most accurate test need to use Ram Drive and have to use powerful machines to illuminate the machine bottle neck factor out.

June 22, 2009 7:43:09 AM

Correction: "eliminate" (sorry)
June 22, 2009 7:46:44 AM

Cat 5e is actually a newer standard than the aging Cat 6 standard. Cat 6a however is a relatively new standard that I would recommend, it does support 10Gb/s networks as well.
Anonymous
June 22, 2009 8:04:56 AM

First of all, gigabit ethernet uses entirely different addressing and encoding from 100-meg, and overhead is one heck of a lot greater than that.
First of all, there's the 10b/8b encoding, so an 8-bit byte is encoded to a 10-bit unit. Then there's a concept of invariable frame sizes, whereit might be possible that a TCP/IP packet spans two frames, filling 100% of the first and 1% of the second, it means 50.5% efficiency. Third, every frame is payload only in part, rest is taken up by header information, footer and CRC. It's not much, perhaps about 5% of the frame, but it can get noticeable.
First, you have to divide by 10, not by 8, to get the speed in bytes/second (ie. 100 MB/s, not 125 MB/s).
Second, if you transmit a lot of inefficient frames (networking programs aren't exactly frugal about bandwidth when they have gigabit ethernet, and next to none are actually optimized in any way for it), you might lose up to half of the bandwidth.
Third, when you factor in the frame level overhead, you might end up with maybe 40-45 MB/s of the promised 100 MB/s...

Fortunately, a lot of these issues can be resolved by optimizing software and firmware to take advantage of the available bandwidth and idiosyncracies of gigabit ethernet.
June 22, 2009 8:14:17 AM

Ok my bad, this article is not for Tomshardware it is not meant for people that understand networking or maybe even computers. Pass this article on to another site with more "normal" visitors.

Testing with a different file for ram to ram then used in the other tests really show the errors in these tests.
June 22, 2009 8:29:09 AM

this is why i am a regular reader here at tom's .
June 22, 2009 8:50:36 AM

What i want to see is the effect of jumboframe packets and hdd allocation unit size (or stripe size) and its effects on network transfers since the packets etc transfer differently across the network cable - benchmarks?
June 22, 2009 9:00:02 AM

For all tech people out there, the title of the article should have been a dead give away
Quote:
Gigabit Ethernet: Dude, Where's My Bandwidth?
about the technical aspect of this piece. Sure they could have used a server platform with a server os, SSD's and ram disks, and why not some tech language what most people don't understand. But, as the titles states in a very suggestive way, this article is for people that have simple questions and seek simple answers.

I'm ok with this piece, it isn't and injustice or it isn't wrong in any way IF you look at who it is addressed to. Remember the KISS rule guys.
June 22, 2009 9:31:53 AM

Technically you could see a 10 fold increase in thruput. A 100Mb/s network is capable of 12.5MB/s thruput. A 1Gb/s network has a thruput of 128MB/s. So therefore 100Mb/s(12.5MB/s) x 10 = 1Gb/s(128MB/s). Of course this takes in to account an optimal or perfect network invironement, one with a single user and a sata based SSD that can push that amount of data on the read/write. Your typical magnetic based HDD running at 7200 RPM will be hard pressed to push more then 100MB/s on the write which is going to be your data bottle neck as you can only push data as fast as it can be written. Of course even at that speed you are around an 8x improvement in thruput. You will also lose bandwith with overhead alone of roughly 10%(on this I could be mistaken). Again this all takes into account a perfect environement to test in.
June 22, 2009 9:33:45 AM

First of all I would like to point out to IronRyan21 that the actual Gigabit ethernet standard is for Cat5. Second I would have preferred to see more through tests done as the "50 ft" of cable used is a very short run for ethernet since the maximum length is actually 100 meters or 328 Ft as spectrewind pointed out. In a home you probably will not see runs much over 50 Ft but the setup used in the article was actually 2 x 25 Ft runs. If the tests were done closer to the maximum length you would then see a much bigger difference when changes were made to how the cable was run and the cable itself. I would want the article to be redone with runs of 50, 100, 200, and 300 Ft to see if the conclusions were correct or if they only apply to very short runs. I would have also liked to have seen testing on the throughput difference using jumbo frames as well as different file sizes.

I think the RAM disk was a good idea to do a maximum throughput test in using real world data copies but that was the only good thing about the article that I can see.
June 22, 2009 9:51:09 AM

@all complaining about the technical aspects of this article: I think the target audience is NOT network administrators.

One the other hand is is worth mentioning, that transfer speed over gigabit network from disk to disk, depends on the files size transferred and number of files transferred.

It's one thing to copy over network a 4 gig file. And is totally different to copy 40k+ files totalizing 4 gigs. For the latest scenario, performance will take another hit due to increased I/O overhead @ disk level.
June 22, 2009 10:21:54 AM

It's a good introduction to some aspects of file transfer. It's probably a question many people wondered about. ram-ram transfer makes the point clear about gigabit capabilities.
June 22, 2009 11:08:17 AM

bin1127It's a good introduction to some aspects of file transfer. It's probably a question many people wondered about. ram-ram transfer makes the point clear about gigabit capabilities.


- on the button for me. Quite why authors of articles such as this are flamed so heavilly is beyond me. If the flamers could only write a full retort that covered their point as fully, there might be some healthier debate on the subject. Jumbo thingamebobs? Please at least take as much time to compose your thoughts as the author of the original article did!

Sorry i'm not an expert on networking. Like many of you out there I'm just putting in a network at home, and this makes enlightening reading.
June 22, 2009 11:28:53 AM

Could you run a test of cabling difference in a more large scale place? ie. does it make a difference on a hp 4108 switch if one was to replace 60 cat5e cables with cat6, or what about transmit errors when running the backbone between two switches on cobber lines hugging thick power cables?

We're using cat6 at work whenever we do something new, but we have plenty of old cat5e cables as well. Not sure if it actually makes a difference.

btw. cat 5 without the e mark does NOT work very well with gigabit. We've got a bunch of those old cables in one of the offices I am working, and we're constantly getting crc/allignment warnings because it's trying to run gigabit over an old cable. Also the meru 320 APs didn't work on legacy cat5 cables - at least not at 60m length.
Anonymous
June 22, 2009 11:59:43 AM

Excellent article.
June 22, 2009 12:30:58 PM

I have to say, you all complain WAY to much. I personally found it quite an informative article. The title was not "how to push your gigabit LAN to the max" after all. This really explained a lot to me about how network file transfers occur and what is involved.

I do believe that a few months back THG said that they were opening up to user submitted articles, so PLEASE, show them how it is supposed to be done....oh wait, you just talk shit, so go sit in your cubicle and shut up please well these writers actually take the time to try helping and explaining these topics to readers. You complain like you are actually paying for this service. Get over yourselves.
June 22, 2009 12:42:48 PM

The ram to ram test is very strange. Using 8b/10b encoding the maximum speed should be 100 MB/s.
June 22, 2009 12:54:26 PM

jankeeThe article does not make any sense and created from an rookie. Remember you will not see a big difference when transfer small amount of data due to some transfer negotiating between network. Try to transfer some 8GB file or folder across, you then see the difference. The same concept like you are trying to race between a honda civic and a ferrari just in a distance of 20 feet away.Hope this is cleared out.

Wow, what a dumbass. Even worse is he tries to compare it to cars. lol
June 22, 2009 1:04:06 PM

Look... The fact is that they screwed up a number of fairly important FACTS. Because of that, the entirety of the article is corrupt.

Gb Ethernet is 119MB/s NOT 125MB/s

PCI is NOT 133MB/s. It is 125.8MB/s(33MHz PCI) or 251.7MB/s (66MHz PCI). The last revision of PCI is typically the 66MHz variant and is on more PCs than you would think. But then, PCI is shared bus so if you have a slower 33MHz card and a faster 66MHz card installed at the same time, the lower speed card sets the pace.

Ethernet max length, as pointed out several times, is 328 feet NOT 100 feet. Also Cat5 IS Gigabit certified so long as your cabling infrastructure was IBDN certified (Obviously not something you will see in a house but is much less an issue for short runs.) Cat6 was a scam and 5e is more than enough for client use. The newer 10Gb over copper standards are not for PCs, they are for server racks. Eventually, this will get to the consumer level but not for a LONG time so newer versions of Cat6 or even Cat7 are a moot point.

And yes, copying a few LARGE files compared to backing up your pictures across the network is a HUGE difference. The exact same issue is seen even going from one drive to another in the same machine. The i/o penalty for opening a file and negotiating and then closing the file that frequently comes with a massive penalty.

All this said, anecdotally, I can say with great assurance, that the average achieved performance with 100Mb Ethernet is about 5MB/s. The average performance on 1000Mb Ethernet is about 30MB/s with large file copies achieving up to 55MB/s. The biggest impact, as pointed out in the article, is the hard drive performance. Which makes me also wish they did a test with an SSD or with an SSD array so that they can show any possible bottlenecks from the SATA interface.

All around, a poorly written/thought out article.
June 22, 2009 1:06:46 PM

I should note that when I say "large file copies achieving up to 55MB/s", I am referring to common household use and not lab conditions.
June 22, 2009 1:32:16 PM

good article and comparisons Mr Woligroski, I would like to see a follow up with SSD's and maybe comparing different routers. See if the El cheapo's perform just as well as name brands. Would be interesting to see. I liked the article.
June 22, 2009 1:52:16 PM

And information on things to do to make transfers faster (as the article intro stated)???
June 22, 2009 1:57:55 PM

i never had to move large files before i added gigabit equipment on my home network...

my file server is just a simple Ubuntu box with bunch of hard drives in it with gigabit ethernet card...i use it for downloading tv episodes on bittorrent network and backing up gaming files...

i have D-Link DGL-4300 linking between my gaming pc with onboard gigabit card...

also i have gigabit switch from Linksys/Cisco to connect Xbox 360, PS3, and Popcorn Hour to the router for online gaming, file sharing, and watching downloaded video files and playing my music collection...

the slowest file transfer is between Popcorn Hour and file server since Popcorn Hour only has 100Mbps interface...moving a large movie file, say DVD iso to play movies from other region, takes about 45 minutes to more than an hour...

but file transfer between my gaming pc and file server is FAST...once i moved a backup DVD iso in a few minutes...
June 22, 2009 2:00:38 PM

Enough about anal specs of ethernet guyz, who really cares which way you twist the wires? or 119 vs 125 bla bla bla. I just wanted to add one more thing Mr Woligroski if and when a follow up article is decided on. Can you also include various MTU/jumbo packet frames sizes and if they make a diff too? Thanks for a job well done sir!
June 22, 2009 2:22:52 PM

I've got a 2 relatively high end units. One with a RAID 6, capable of faster than 200mb/s write and faster than 400mb/s write. Vista 64 to linux 64. CAT 10gx wire. Mainly,Intel Server PCI-e nics. Latest version of software(2.6.30) and patches. From server to Vista 6mb/s and Vista to server 15mb/s. Any suggestions?? (Besides switching linux to Windows?) BTW Vista unit has 3 SSD's.
June 22, 2009 2:29:51 PM

I found it to be an informative article. Just last week I ran Cat5e to the living room and was a little disappointed by the overall speed. Granted night and day better than 54g for streaming media - but still not as fast as I would have hoped. I will probably do a bit of "testing" on my own just for curiosity sake.
June 22, 2009 2:41:33 PM

I set up gigabit between my two PCs a few years back, and even after a lot of tweaking I found it was faster to use cold-swappable hard drives (the cheap kits from CompUSA) instead. The shutdown/restart time was faster than the transfers.

I'd certainly go back to gigabit if I wired my house, but 802.11N is doing just fine and avoids paying someone to crawl through my attic. Meanwhile, my new Linksys WRT610N does show the value of gigabit routers in that the 'speed boost' thing that RoadRunner offers only showed up when I had a router fast enough to catch it. SHORT burts of 1.2mbps downlaods are better than none at all...
June 22, 2009 2:50:47 PM

Good article. Ignore the nitpicking of details. It explained the following:
1) Wny you might not get 10x transfer speeds with a gigabit network.
2) Identified the hard drive as the primary limiting factor.
3) Showed that cabling has no real impact on performance.
June 22, 2009 2:51:25 PM

I would like to have seen the importance of flow control in routers/switches and its impact on throughput mentioned. Device firmware is generally written to ensure availability of the network and not throughput. They will not allow any single client to destroy network performance for everyone else by using all the bandwidth. As a result, it is extraordinarily difficult to get full gigabit throughput through any device. It also turns out there is no standardized behavior when flow control is invoked. That is, the amount of time spent waiting depends solely on the whim of the firmware engineer. That means the throughput varies dramatically based vendor and even combination of devices on a network.

I have personally tested around a dozen consumer gigabit switches for throughput. The best performance thus far has been the DGS-2205 or DGS-2208. Both will allow for full gigabit throughput using jumbo frames. Ironically, it's probably a bug in their firmware as that behavior is detrimental to the network at large. Anyway, buyers beware. Gigabit is rarely gigabit.
June 22, 2009 2:56:36 PM

The guys over at Small Net Builder did some great testing to see how fast of a small network you could build in regards to file transfer speed. They explored more hardware and OS options, though the final answer was basically the same. System RAM plays a large part in file transfer efficiency. If I remember right, they found that Vista will buffer a file transfer up to the amount of available system RAM, so for a 32-bit OS, that's usually 2 GB of available (NOT total). 64-bit OS you can obviously go much higher. 100 MB/sec might be achievable as long as your file transfer isn't larger than free RAM. Once you go over that, it drops down to the bandwidth the disks can supply, just as this article says.
June 22, 2009 2:58:48 PM

Wait, I thought this was common knowledge.. though judging by some of the questions and comments people ask, I guess it wasn't. I agree with most of the harsher comments, though I do appreciate and value the results.

But seriously, a router isn't better than a switch because of the processing overhead. At my office we have about 4 routers, mostly IP assignment, but all the processing is done by 12 Catalyst 6500's. That's because each one has 2 terabytes per second in throughput. A switch is designed just to transfer data and it is the most efficient path of doing so. You want a fast network? Get a managed switch.
June 22, 2009 3:07:35 PM

raptor550 makes a great point, and I've been looking for ways to speed up my own home network. I use a router for DHCP assignment, and to supply wireless access. Under high traffic, it lights up like a Christmas tree and speeds start to suffer. I just picked up a small 8 port gigabit Smart switch, though my question now is what to do for DHCP? Use the old router, or dedicate a server to manage it?
June 22, 2009 3:13:38 PM


OK - I found this article to be good and informative at the level it was intended. I also am very interested in a deeper dive on the gigabit networking.

I have regular cat5 run throughout my house up walls through the attic down walls etc. Do a cabling test like another poster recommended at 50 foot intervals to 400 feet (yes I know 328 is a max). Be interesting to see what happens. A lot of us want to know what our old cat5 layer is really doing to us.

I'd also like to see some nice fat drive arrays and ssds in the picture.

Overall nice article, now let's get the in depth geek test!



June 22, 2009 3:25:02 PM

Good article to get things started. Sometimes you want a technical manual, other times you look at the quick start guide. This was the quick start guide.

"A switch is designed just to transfer data and it is the most efficient path of doing so. You want a fast network? Get a managed switch."

Nah, if you're that hard core between your PC and your server, just get a 15ft Cat5e (or 6) crossover cable, 3xRaid SSD's or new HDD's and go nuts tweaking your dedicated NICs.
June 22, 2009 3:31:33 PM

Wathman... I use a little gigabit switch and a PC running Clarkconnect for my Router/Firewall/DHCP Server/File Server/Web Server/Print Server. I noticed a farily significant increase in my internet throughput after going with this setup as apposed to my little cheapie router. My speakeasy.net speedtest went from around 1.5mbps to around 12mbps. I'm sure this wouldn't be an issue if I had a better router to begin with since most of the good ones can throughput over 100mbps anyway. You're probably ok to use your old router for DHCP and internet access, but a Linux router would give you more options and might be a little faster.
June 22, 2009 3:37:00 PM

i'm pretty neutral for this article since i've read (forgot where, here at TH probably) long ago when gigabit products came out, it is found that it was overkill for home networks. from then on, i thought gigabit ethernet was for SOHO networks with busy multiple PCs, add to that the cost of the switch.
my home network is mostly internet and media streaming traffic.
anyways, i agree with comments to include some technical aspects so regular readers can learn something new.
i know, uninterested readers can easily skip those pages.

i have a question though, what happens if an ordinary router serves internet to a gigabit switch? PC to PC file transfer remain gigabit speeds, right???
Anonymous
June 22, 2009 4:00:24 PM

I'm wondering why they used the smaller and slower hdds and yet the conclusion page show the same WD 640GB hdd I have that tests out on my system at about 105MB/s. That dive is about 50% full. My new WD 1TB hdd averages 108 MB/s and my Velociraptor tops 120 MB/s so I hope they re-run these tests with newer and faster hdd.
June 22, 2009 4:03:32 PM

Guys, get off the cleeve's back. He did a test of real-world results and even if every technical detail he wrote might not be 100% correct, he demonstrated what a home network might consist of, and how it would perform. I appreciate the simplistic approach of actual performance rather than the nitty gritty details of network interaction.
Anonymous
June 22, 2009 4:09:59 PM

What about error correction and maybe and enginering channel takes up the last 14mb's stoping the tests from hitting 125?
June 22, 2009 4:15:06 PM

Computer_Lots, thanks for your benchmarks. After I get to tweaking, I'll have a better idea of what I should be getting for performance. I also have a PC running Untangle that I could bring back up online to use in a similar fashion as you use your Clarkconnect PC. Now that I have a new smart switch coming in, I might have better luck with the Untangle server.

zodiacfml, for the most part that is correct. Also keep in mind you could tweak the performance a bit if you use hardware that supports jumbo frames, though jumbo frames is not a feature that will work miracles. Expect performance gains in the neighborhood of 5-10%. As for internet connections through routers, don't worry about gigabit connections between your modem and router. Unless you have a brand new DOCSIS 3.0 cable modem, or whatever hardware needed for Verizon FIOS, your modem is only capable of a 100 Mbit ethernet link to the router.
!