10GBPS internet + NIC/thunderbolt

firehawk_1 · Mar 15, 2018

So, I can get 1GBPS internet from Comcast and I expect to maybe get around 800MBPS min. In fact, it was proven however the issue became that the consumer infrastructure is not quite ready.

My equipment and quality is more than the average Joe due to the nature of my work.

IT was proven that we can get 1GBPS by using thunderbolt connection instead of direct NIC (the NIC is 1GBPS).

To cut the long story short, if I want a NIC, is it better to get a dual 10GBPS NIC? Will it give me 1GBPS like the thunderbolt connection? Again, I don't expect the full 1GBPS but in that region.

I don't want to purchase something and not be able to return it, since it can be expensive.

Also, what routers (not modem) can be recommended to support bonding as well as having the ability for:

- Wireless Mac filtering
- DYNDNS/DDNS

My E4200 is GREAT but does not support 10GBPS ports, only 10/100/1000 and has 4 ports which are all in use so would not be able to do bonding if needed.

Thank you.

bill001g · Mar 16, 2018

Part of the issue is you are using a router that is almost 7yrs old. Newer routers can get around 900mbps, there is overhead so you can never gigabit.

To get those fast speeds they have special NAT acceleration hardware in the devices. Older routers used the CPU chip and were limited by the cpu. The big problem is the hardware accelerator bypasses the cpu so you lose a lot of the fancy functions. Even simple things like utilization screens don't work. I know a lot of the firewall feature also will not work because the data does not pass the cpu. If you turn those off your speed will drop to maybe 300-400 depending on how fast the cpu in the router is.

This list should get you a few options to try

https://www.smallnetbuilder.com/tools/charts/router/bar/179-wan-to-lan-tcp/35

Port bonding does not work the way you think. The standard 802.3ad (lacp) at best can load balance by session which means a single transfer will still only use a single cable.

You can get routers with 10g ports but again the problem is the CPU. If you really wanted to do this you are best taking a dual nic pc and building your own.

But you can get a bunch of routers fairly inexpensively that can run very close to 1g.

In most cases it is not going to be your connection that is the limitation. Many internet sites have artificial limits placed on their connections to prevent a small number of users with very fast internet from bogging down the system for everyone.

BFG-9000 · Mar 16, 2018

For years now, if you ordered 2Gbit service from Comcast, the modem had a 10Gbit port for one desktop and a regular 1Gbit port you were supposed to plug the provided Wifi router into.

As only a PC can get you enough horsepower to cheaply rout that fast, a dedicated PC running something like pfSense with at least a 2-port 10Gbit NIC could be used as the router. Unmanaged 10Gbit switches have really come down in price but it's still cheaper to get cards with more ports.

At only 1Gbit, many cheap consumer grade routers now exist that can do WAN-LAN approaching that number. Note though, that using fancy things like QoS will drop things into pure software mode which is entirely dependent on CPU. For example your old MIPs CPU router at just 480MHz can only rout in software at ~70Mbps with extensive QoS rules including fq_codel, or about 100Mbps with something simpler like Tomato's Bandwidth Limiter for controlling Bufferbloat.

With the original firmware and no features enabled, it tested at 687Mbps WAN-to-LAN. LAN-to-LAN is of course at full wire speed. If you don't care about SambaCry, DNSmasq vulnerabilities or obviously KRACK then you could just run that.

Given that support for your router has ended and 3rd party firmware like Tomato or DD-WRT pretty much only run in software nowadays, you can see that your ISP speed is some 10x too fast for that old router.

Port bonding is only good for multiple simultaneous users, as any one user will only see the bandwidth of 1 port. It's good for failover too, in case 1 port dies. Hilariously though, many routers that advertise bonding will do it in software using the CPU, so that 2 bonded ports end up slower than a single one.

firehawk_1 · Mar 16, 2018

Thanks all.

So here is the thing:

the actual cable modem is a new one, supports DOCSIS 3.1 (Motorola MB6800 and on their approved list). The 2nd engineer that came out in the day said to me that he has seen this issue ALOT in the field and has no answer as to why this happens in terms of not getting even more than 400MBPS!

He did a speed check and can definitely get about 869MBPS! This was using their diagnostic hardware but also brought along his own personal laptop and was connecting using the thunderbolt connection on his MacBook I believe.

So my cable modem connects to my E4200 router which then connects to my computer (and other equip on the LAN).

So I am still confused (great explanation btw to your responses!).

What should I do to get around that speed? Or even at least more than 500MBPS since, even though it was a difficult confirmation, I know I can get around the speed the engineer was able to get and I CAN get 1GBPS package.

We were using speedtests as our basis for confirmation (same website) and ran it many many times with different methods (i.e E4200 to modem, or modem direct plugged into the computer) but always had issues where the max I could get was 300-350MBPS (on 1GBPS NIC Port)

In terms of the wireless router (E4200), if it does come to it (not fully convinced that is the issue), then which ones would you recommend that has what I need? (i.e wireless MAC filtering and DDNS support - essentially very close to the admin features as the E4200 or better)?

My "stripped down" test basis will be the cable modem direct to the computer so somewhere there is a slowdown, and definitely not the computer configuration either.

bill001g · Mar 16, 2018

I am not sure how you connected thunderbolt, I don't think the modem has that type of connection. If there is a thunderbolt to ethernet converter then it is still ethernet so it should not be a issue. Do not get side tracked, ethernet can easily run in the high 900mbps range.

If you are having problems getting the speed with your pc directly hooked to the modem but the ISP had no issues then I would suspect the PC as being the bottleneck.

You have to be very careful about running too much trash on your pc when you have a very fast internet connection. In particular any of those software network "game" accelerator software cause bottlenecks. The worst offender is killer nics you need to actually load different drivers to get rid of the feature. Someone just posted the other day that his killer chipset pc was being bottlenecked by this software to under 400mbps. Intel has gotten in on this game also but in that case you just need to be sure you do not have the program running the driver itself is ok.

Mac filtering you might be able to do since that is done by the wifi chips. DDNS i suspect will require you to disable the hardware acceleration feature...I don't know for sure since manufactures do not document much about the hardware acceleration and few people have more than 200mbps internet and not affected.

To a point I am surprised that cheap routers can even run as fast as they do. The problem always has been the NAT which requires the hardware to modify every packet and recalculate the header checksums. Somehow the manufacture have moved this to ASIC based hardware...and they are very secretive about how they do it. ASIC are much faster than general CPU but they can not really do complex functions. So it is a trade off you get fast but lose fancy features....unless you brute force the issue with a large cpu that needs something other than a consumer router.

BFG-9000 · Mar 16, 2018

Broadcom's bcm_NAT actually uses the switch chip to help do some of the routing, which is pretty darned elegant. That's why the bandwidth meters and such are inaccurate when the hardware acceleration is enabled--many of the packets never even reach the routing CPU. And of course this means QoS rules cannot be applied to those packets.

The last time I had a gigabit setup only transferring 300Mbps I was using Pentium 4s and those horrible and hot 2-chip gigabit NICs like National Semiconductor or DL-2000. There are actually a lot of settings that can slow things down--if you've toggled on any of the optional offload engines for example, it may use less CPU, but performance is also lower because the card is so much slower than a modern CPU. So set everything back to defaults. And as bill001g mentioned, "Killer" drivers can be so bad that loading the Realtek drivers instead to bypass its extra CPU entirely can often speed things up.

You'll need two gigabit PCs on the network to check if they can even transmit or receive faster than that. Used to be you'd need to setup a couple ramdrives but modern SSDs are plenty fast enough to saturate gigabit. Load up your favorite LAN speed tester and see what you get.

firehawk_1 · Mar 16, 2018

thanks again for your enlightening response.

I do work in the IT field myself (mainly software) so I know only so much in relation to hardware. I still don't think its my desktop computer limiting the speed (I did build it and all that, and no crap/junk stuff on the system) but wish I could believe it was.

just FYI, it is using the onboard NIC which is:

Marvell Yukon 88E8056 PCI-E Gigabit Ethernet controller.

I also had the same issue on my laptop (far newer than the desktop) and the laptop I got about 3 years ago now (yup, everything - even my desktop uses SSD drives).

I think he plugged the modem to his laptop using the normal Ethernet jack - I got confused when he said he plugged it into the thunderbolt but said that he has a thunderbolt to Ethernet adapter in his truck for me to try (which I did - same issue).

So yup, he plugged it into his MacBook, and his MacBook I think was 2012 or 2014?

and FYI on the local lan, I can definitely do file transfers and get around 700-800MBPS transfer rate (wired) going through the 1GBPS ports on all systems.

BFG-9000 · Mar 16, 2018

That is quite an old NIC and the newest drivers appear to be from 2012. Most notably it can't offload anything for iPv6 (there's no such thing as an IP header checksum at all in iPv6 so that can only be offloaded in iPv4, but TCP and UDP pseudo-headers are payload data in iPv6 for which checksums must be calculated--it's no longer optional for UDP due to the lack of an IP header checksum).

What's the CPU usage look like while you run the internet speedtest, and what happens if you disable iPv6?

firehawk_1 · Mar 16, 2018

old NIC indeed!

CPU usage is very very low. Maybe just under 1%?
I have a core i7 extreme 3.46GHz CPU.

I do get an IPv6 address and when I disable it, same speed really. (FYI, the router (E4200) does support and has IPv6 address from the ISP too as well as IPv4)

bill001g · Mar 16, 2018

Maybe its something related to speed test. You can run a old tool called iperf. There are a couple public iperf servers you can transfer data from. This is a very simple command mode test it eliminates a lot of the stuff related to the OS and the browser. I will assume you are not running the flash version of speedtest, that one the flash limits the speed.

firehawk_1 · Mar 16, 2018

Correct - not using the flash based test.
But the speedtest, we ran it multiple times, different browser sessions too and on their own devices (the field test device which actually uses speedtest and their laptop too). He was able to get the high speeds. :-/

firehawk_1 · Mar 18, 2018

OK so...

I ordered a 10GB NIC (Asus) for tomorrow. I wanted a dual 10GB NIC but decided not to, for now.
I have a 10GB Switch also.

Still have the E4200 router but... question is:

I know I can do a test to see if the modem, plugged in directly to the computer, will give me 800MBPS+ speed but in the event that it DOES, what does one need to do to get the same/close to speed by using a router?

Would I need a wireless router that has a 10GBPS WAN port or is 1GB ok to get those speeds?

firehawk_1 · Mar 19, 2018

ok,

got the 10GB NIC, all installed.
Direct wire to the modem

still not even getting 500+!

what the heck?

BFG-9000 · Mar 20, 2018

Time to try a different OS. Boot from a Live distro and hopefully it includes drivers for the Marvell. You'd have to install the Aquantia linux drivers for the ASUS.

The driver situation in linux can be rather iffy compared to the walled garden of Unix that is MacOS, but if it works fine then at least we can isolate the problem to your current Windows install.

firehawk_1 · Mar 20, 2018

Well, I have tried it on 2 different devices with 2 different versions of Windows (Win10 and Windows Server 2008R2) and its the same result :-/

failboat · Mar 20, 2018

Replace your equipment thats outdated with newer 1Gb, it shouldn't break the bank. Going up to 10, very costly, or using bonding will only help heavy LAN traffic. You can build a Pfsense router if you are really worried about speed and not about money. A Ubiquiti Edge router is fast as well. Pair it with a lvl2 switch and your LAN traffic won't hit the router. All in one router solutions may not fit your needs. Adding in wireless access points will be helpful as well. Let the router handle the minimum.

At the end of the day if your ISP is under serving your area it won't matter what you buy. You get what they give.

firehawk_1 · Mar 20, 2018

So everything is already 1GB and has been for quite some time.
LAN traffic is not the issue (btw, seems like im saying "no no no" to everything, but it truly is not the case). I have an IP phone, a desktop computer, a laptop and most of the heavy stuff is on the desktop computer.
The wireless does not consume much data at all but at this moment in time, it's directly an issue with either the provider or modem. I don't think it's the modem - it's brand new and on their "compat" list.

Got the engineer coming out again tomorrow. Let's see what this guy says. But it is frustrating.

failboat · Mar 20, 2018

firehawk_1 :

It sounds like you have been able to get max speeds in some of your tests. If the max speeds are consistent using the same test parameters then the weak link has to be down the line.

Have you tested different direct to modem connections? If you are seeing any difference across devices it's an issue on your side. If you add a router in between and performance drops, it's going to be the router. Don't take a 1Gb label at face value.

firehawk_1 · Mar 20, 2018

Sure, I know. I expect around 800-900MBPS, nothing less to be honest. It's definitely something on their end and finding it seems to be a PITA. Even the engineer said that he sees this all the time and saw maybe twice, some people get around 900MBPS and is possible. :-/

10GBPS internet + NIC/thunderbolt

Distinguished

Titan

Splendid

Distinguished

Titan

Splendid

Distinguished

Splendid

Distinguished

Titan

Distinguished

Distinguished

Distinguished

Splendid

Distinguished

Splendid

Distinguished

Splendid

Distinguished

Share this page