Gigabit Ethernet: Dude, Where's My Bandwidth?

Name: Gigabit Ethernet: Dude, Where&#039;s My Bandwidth?
Author: Don Woligroski

by Don Woligroski June 22, 2009 at 1:00 AM

I was in no hurry to upgrade my home network from 100 Mb/s to gigabit speed, which is odd when you consider how much time I spend waiting for file transfers. That's because when I spend money on PC upgrades, I think of the components that offer an immediate performance increase in the applications and games that I run. Putting cash towards things like video cards, CPUs, and even peripherals is almost like buying toys for myself. For some reason, network components don’t inspire the same amount of excitement. Indeed, it’s harder to let go of hard-earned money when it feels like an infrastructure investment rather than a self-gifted advance birthday present.

Inevitably, however, my high-bandwidth networking demands were making it obvious that 100 Mb/s weren’t going to cut it anymore if I valued my time. All of the systems I was running at home already had gigabit network controllers built into their motherboards, and I remember looking into the rest of the hardware shopping list that I needed to step up my network to full gigabit speed.

No, a gigabit network does not have to be this complex

When I was all done collecting the pieces, I remember copying a large file over the old 100 megabit equipment, which took about a minute and a half, and then upgrading to the gigabit network. After the upgrade, it took about 40 seconds to copy the same file. It was a nice performance boost, but not quite the 10 times difference between 100 Mb/s and 1 Gb/s I was expecting.

What's with that, anyway?

Let's find out what's going on by examining the basics. If you're a network wizard and you know all about MTUs, frame sizes, and the minutia of optimizing a network, this article will be somewhat elementary reading. But if you don't know much about networking except that there are computers and cables involved, read on. We’ll be going over the fundamentals of gigabit networking, the variables that will impact the network speed, and what a home user can do about them to get the most out of Gigabit Ethernet.

Summary

About the author

Don Woligroski

This thread is closed for comments

125 comments

Comment from the forums

Your comment

gwiz1987

why is the RAM-to-RAM network max speed on the graph 111.2 when u state 111.4? typo?
drtebi

Interesting article, thank you. I wonder how a hardware based RAID 5 would perform on a gigabit network compared to a RAID 1?
Hello

Thanks for the article. But I would like to ask how is the transfer speed measured. If it is just the (size of the file)/(a time needed for a tranfer) you are probably comsuming all the bandwith, beacuse you have to count in all the control part of the data packet (ethernet header, IP headrer, TCP header...)

Blake
jankee

The article does not make any sense and created from an rookie. Remember you will not see a big difference when transfer small amount of data due to some transfer negotiating between network. Try to transfer some 8GB file or folder across, you then see the difference. The same concept like you are trying to race between a honda civic and a ferrari just in a distance of 20 feet away.

Hope this is cleared out.
spectrewind

Don Woligroski has some incorrect information, which invalidates this whole article. He should be writing about hard drives and mainboard bus information transfers. This article is entirely misleading.

For example: "Cat 5e cables are only certified for 100 ft. lengths"
This is incorrect. 100 meters (or 328 feet) maximum official segment length.

Did I miss the section on MTU and data frame sizes. Segment? Jumbo frames? 1500 vs. 9000 for consumer devices? Fragmentation? TIA/EIA? These words and terms should have occurred in this article, but were omitted.

Worthless writing. THG *used* to be better than this.
IronRyan21

Quote:

There is a common misconception out there that gigabit networks require Category 5e class cable, but actually, even the older Cat 5 cable is gigabit-capable.

Really? I thought Cat 5 wasn't gigabit capable? In fact cat 6 was the only way to go gigabit.
cg0def

why didn't you test SSD performance? It's quite a hot topic and I'm sure a lot of people would like to know if it will in fact improve network performance. I can venture a guess but it'll be entirely theoretical.
MartenKL

Gbit is actually 10^9 bits per second, ie about 119 MB/s.
flinxsl

do you have any engineers on your staff that understand how this stuff works?? when you transfer some bits of data over a network, you don't just shoot the bits directly, they are sent in something called packets. Each packet contains control bits as overhead, which count toward the 125 Mbps limit, but don't count as data bits.

11% loss due to negotiation and overhead on a network link is about ballpark for a home test.
jankee

After carefully read the article. I believe this is not a tech review, just a concern from a newbie because he does not understand much about all external factor of data transfer. All his simple thought is 1000 is ten time of 100 Mbs and expect have to be 10 time faster.

Anyway, many difference factors will affect the transfer speed. The most accurate test need to use Ram Drive and have to use powerful machines to illuminate the machine bottle neck factor out.
jankee

Correction: "eliminate" (sorry)
MartenKL

Cat 5e is actually a newer standard than the aging Cat 6 standard. Cat 6a however is a relatively new standard that I would recommend, it does support 10Gb/s networks as well.
First of all, gigabit ethernet uses entirely different addressing and encoding from 100-meg, and overhead is one heck of a lot greater than that.
First of all, there's the 10b/8b encoding, so an 8-bit byte is encoded to a 10-bit unit. Then there's a concept of invariable frame sizes, whereit might be possible that a TCP/IP packet spans two frames, filling 100% of the first and 1% of the second, it means 50.5% efficiency. Third, every frame is payload only in part, rest is taken up by header information, footer and CRC. It's not much, perhaps about 5% of the frame, but it can get noticeable.
First, you have to divide by 10, not by 8, to get the speed in bytes/second (ie. 100 MB/s, not 125 MB/s).
Second, if you transmit a lot of inefficient frames (networking programs aren't exactly frugal about bandwidth when they have gigabit ethernet, and next to none are actually optimized in any way for it), you might lose up to half of the bandwidth.
Third, when you factor in the frame level overhead, you might end up with maybe 40-45 MB/s of the promised 100 MB/s...

Fortunately, a lot of these issues can be resolved by optimizing software and firmware to take advantage of the available bandwidth and idiosyncracies of gigabit ethernet.
MartenKL

Ok my bad, this article is not for Tomshardware it is not meant for people that understand networking or maybe even computers. Pass this article on to another site with more "normal" visitors.

Testing with a different file for ram to ram then used in the other tests really show the errors in these tests.
cyberkuberiah

this is why i am a regular reader here at tom's .
apache_lives

What i want to see is the effect of jumboframe packets and hdd allocation unit size (or stripe size) and its effects on network transfers since the packets etc transfer differently across the network cable - benchmarks?
SpadeM

For all tech people out there, the title of the article should have been a dead give away

Quote:

Gigabit Ethernet: Dude, Where's My Bandwidth?

about the technical aspect of this piece. Sure they could have used a server platform with a server os, SSD's and ram disks, and why not some tech language what most people don't understand. But, as the titles states in a very suggestive way, this article is for people that have simple questions and seek simple answers.

I'm ok with this piece, it isn't and injustice or it isn't wrong in any way IF you look at who it is addressed to. Remember the KISS rule guys.
Psycomo

Technically you could see a 10 fold increase in thruput. A 100Mb/s network is capable of 12.5MB/s thruput. A 1Gb/s network has a thruput of 128MB/s. So therefore 100Mb/s(12.5MB/s) x 10 = 1Gb/s(128MB/s). Of course this takes in to account an optimal or perfect network invironement, one with a single user and a sata based SSD that can push that amount of data on the read/write. Your typical magnetic based HDD running at 7200 RPM will be hard pressed to push more then 100MB/s on the write which is going to be your data bottle neck as you can only push data as fast as it can be written. Of course even at that speed you are around an 8x improvement in thruput. You will also lose bandwith with overhead alone of roughly 10%(on this I could be mistaken). Again this all takes into account a perfect environement to test in.
thexder1

First of all I would like to point out to IronRyan21 that the actual Gigabit ethernet standard is for Cat5. Second I would have preferred to see more through tests done as the "50 ft" of cable used is a very short run for ethernet since the maximum length is actually 100 meters or 328 Ft as spectrewind pointed out. In a home you probably will not see runs much over 50 Ft but the setup used in the article was actually 2 x 25 Ft runs. If the tests were done closer to the maximum length you would then see a much bigger difference when changes were made to how the cable was run and the cable itself. I would want the article to be redone with runs of 50, 100, 200, and 300 Ft to see if the conclusions were correct or if they only apply to very short runs. I would have also liked to have seen testing on the throughput difference using jumbo frames as well as different file sizes.

I think the RAM disk was a good idea to do a maximum throughput test in using real world data copies but that was the only good thing about the article that I can see.
zetone

@all complaining about the technical aspects of this article: I think the target audience is NOT network administrators.

One the other hand is is worth mentioning, that transfer speed over gigabit network from disk to disk, depends on the files size transferred and number of files transferred.

It's one thing to copy over network a 4 gig file. And is totally different to copy 40k+ files totalizing 4 gigs. For the latest scenario, performance will take another hit due to increased I/O overhead @ disk level.
bin1127

It's a good introduction to some aspects of file transfer. It's probably a question many people wondered about. ram-ram transfer makes the point clear about gigabit capabilities.
Tattysnuc

bin1127It's a good introduction to some aspects of file transfer. It's probably a question many people wondered about. ram-ram transfer makes the point clear about gigabit capabilities.

- on the button for me. Quite why authors of articles such as this are flamed so heavilly is beyond me. If the flamers could only write a full retort that covered their point as fully, there might be some healthier debate on the subject. Jumbo thingamebobs? Please at least take as much time to compose your thoughts as the author of the original article did!

Sorry i'm not an expert on networking. Like many of you out there I'm just putting in a network at home, and this makes enlightening reading.
neiroatopelcc

Could you run a test of cabling difference in a more large scale place? ie. does it make a difference on a hp 4108 switch if one was to replace 60 cat5e cables with cat6, or what about transmit errors when running the backbone between two switches on cobber lines hugging thick power cables?

We're using cat6 at work whenever we do something new, but we have plenty of old cat5e cables as well. Not sure if it actually makes a difference.

btw. cat 5 without the e mark does NOT work very well with gigabit. We've got a bunch of those old cables in one of the offices I am working, and we're constantly getting crc/allignment warnings because it's trying to run gigabit over an old cable. Also the meru 320 APs didn't work on legacy cat5 cables - at least not at 60m length.
Excellent article.