Possibly stupid question re: 24-port switch, 1 router, and throughput

[BrianDMG]

I'm a long-time reader of tom's hardware, but I joined just to ask this question, because I'm having a hard time wrapping my head around it (and possibly over-thinking it).

When I moved into my current house, I wired the whole place with Cat6 that leads to a 24-port Gb patch panel, which then goes to a 24-port unmanaged Gb switch, which then goes to my Netgear R7000 router (and then out to my modem).

My question is this: My router (a gigabit router) has 4 LAN ports on the back. I only have one Cat6 cable (using only 1 LAN port, obviously) going from the router to the switch. Does this mean that the combined throughput of all the devices in my house is limited to 1Gb? If so, is it possible to use all 4 LAN ports (by plugging more cables from the router to the switch) simultaneously with an unmanaged switch, somehow? If so, would that actually even increase my "total" throughput?

I highly appreciate any insight anyone might be able to offer.

 

[WyomingKnott]

The computers in you house can communicate with each other at gigabit speeds. Gigabit PC to patch panel to Gigabit switch to patch panel to Gigabit PC.

However, their Internet traffic speed is limited by the speed of the connection from the router to your ISP. For a normal consumer connection, this is slower than a single Ethernet connection. So adding a "fatter pipe" between the router and the switch won't do you any good; the trickle is limited by the pipe to your house.

Does that make sense? I find myself reaching for a whiteboard marker to draw a picture.

 

[BrianDMG]

The computers in you house can communicate with each other at gigabit speeds. Gigabit PC to patch panel to Gigabit switch to patch panel to Gigabit PC.

However, their Internet traffic speed is limited by the speed of the connection from the router to your ISP. For a normal consumer connection, this is slower than a single Ethernet connection. So adding a "fatter pipe" between the router and the switch won't do you any good; the trickle is limited by the pipe to your house.

Does that make sense? I find myself reaching for a whiteboard marker to draw a picture.

Thanks the the reply, but perhaps I should have been more specific. I'm not concerned at all about my bandwidth externally to my ISP at all (I'm aware that that speed is determined by my ISP and how deep my pockets are) - I'm specifically asking about LAN speeds. I'll draft up a quick "whiteboard" diagram of my setup and post it in a few minutes.

Here's the diagram:

ETA: Network Diagram

 

[WyomingKnott]

"The computers in you house can communicate with each other at gigabit speeds. Gigabit PC to patch panel to Gigabit switch to patch panel to Gigabit PC." I meant that your lan speeds will be Gigabit device-to-device when the devices are attached to your switch. You'll be able to stream whatever you want, share files with a central server, and so forth at full speeds.

For everything else, the limiting factor (web, WiFi) is in the medium, not in your setup. Your router, for example, can't push data to WiFi clients faster than that data is provided by the single Ethernet connection to the switch.

(your description was good enough that I got the right image in my head, but that's a good diagram).

I wired my house with cat5, and put a cable tray in the garage to neaten some of the mess. Still, my service loops were an ugly disaster.

 

[BrianDMG]

Great, that's what I needed to know (and what I figured), so thanks for that! The second part of my question is how can I make use of the other 3 LAN ports on my router so that my combined throughput would be 4Gb instead of 1Gb? I feel like I already know that the unmanaged switch is going to be the limiting factor here, as I've already tried plugging a second cable from the router to the switch, and it basically brought my whole network down (specifically, it killed my Internet connection, but I still had limited connectivity to internal devices), but I was hoping there was some other way that I hadn't considered. I'm a networking student, so I'm already kind of embarrassed that I don't know the answer. Thanks again for your replies!

 

[WyomingKnott]

I don't see any advantage in channel-bonding here; your hardware probably won't handle it anyway. Nothing on the router-wifi-modem side of that link is capable of aggregate gigabit speed, so nothing is limited by the connection from switch to router. Unless you hang more devices off the router's other ports, and that would take up the router's other ports anyway.

Leave things as they are. There are no cabling related bottlenecks. IMHO.

 

[bill001g]

You got a good lesson in why you must have spanning tree in a network. Cabling it like you did introduced a loop and broadcast packets will wipe out any device connected to any switch port. Consumer grade equipment many times does not support spanning tree so you immediately get a almost dead network when you cable a loop.

As stated both your switch and router must support port bonding more specifically 802.3ad if you want more than 1g of bandwdith. Still even if you were to accomplish this a single session can not use more than 1 path. You would have to have multiple machines talking too each other on both end to exceed 1g of usage. It doesn't really matter since your wireless can come nowhere near even 1g and you might as well plug machine directly into the large switch rather than plug them into the router and have it forward the traffic to the switch.

 

[WyomingKnott]

I'm old enough to remember when Bitnet and Decnet topology (and others, I am sure, but those are the two that I knew) added redundant paths as a major feature, eliminating the necessity of a simple tree architecture. Radical idea at the time.

Why would this be a spanning tree - there is no underlying set of edges to choose from, only the nodes?

 

[bill001g]

You have 2 paths between the switches. Broadcast traffic is send out all ports except the one it was received on. So if you have 2 switches hooked port 4 to port 4 and port 5 to port 5 . So say switch 1 receives a packet from switch 2 on port 4. It sends it out ever other active port including port 5. Now switch 2 get the packet on port 5 and sends it out every active port including again port 4. Traffic goes round and round and on each pass it spams it to every port on the switch. Since there are a lot of broadcast packets on a network it does not take long to cascade into every port on all the switches being pounded.

This is the simplest spanning tree loop you can get...other than plugging a 2 ports together on the same switch. You can only have a single path active between any group of switches and switches that support spanning tree can ensure there is only 1 path.

When you run link aggregation it in effect disables the spanning tree at the port level and treats the bonded group as a single connection. You have to be very careful setting up link aggregation especially using some the automated LACP options you can get short term spanning tree loops as it negotiates the bonding.

 

[BrianDMG]

You got a good lesson in why you must have spanning tree in a network. Cabling it like you did introduced a loop and broadcast packets will wipe out any device connected to any switch port. Consumer grade equipment many times does not support spanning tree so you immediately get a almost dead network when you cable a loop.

As stated both your switch and router must support port bonding more specifically 802.3ad if you want more than 1g of bandwdith. Still even if you were to accomplish this a single session can not use more than 1 path. You would have to have multiple machines talking too each other on both end to exceed 1g of usage. It doesn't really matter since your wireless can come nowhere near even 1g and you might as well plug machine directly into the large switch rather than plug them into the router and have it forward the traffic to the switch.

I get what you're saying, but I just wanted to point out that wireless doesn't enter into the equation with the question I'm asking. I'm specifically asking about wired connections in my LAN. Also, the only cable connected to the router is the single Cat6 going to the big switch, all other machines are on wired connections via keystone jacks throughout the house (which lead to the patch panel, then to the switch).

Also, thanks again for your continued advice, WyomingKnott.

 

[bill001g]

You got a good lesson in why you must have spanning tree in a network. Cabling it like you did introduced a loop and broadcast packets will wipe out any device connected to any switch port. Consumer grade equipment many times does not support spanning tree so you immediately get a almost dead network when you cable a loop.

As stated both your switch and router must support port bonding more specifically 802.3ad if you want more than 1g of bandwdith. Still even if you were to accomplish this a single session can not use more than 1 path. You would have to have multiple machines talking too each other on both end to exceed 1g of usage. It doesn't really matter since your wireless can come nowhere near even 1g and you might as well plug machine directly into the large switch rather than plug them into the router and have it forward the traffic to the switch.

I get what you're saying, but I just wanted to point out that wireless doesn't enter into the equation with the question I'm asking. I'm specifically asking about wired connections in my LAN. Also, the only cable connected to the router is the single Cat6 going to the big switch, all other machines are on wired connections via keystone jacks throughout the house (which lead to the patch panel, then to the switch).

Also, thanks again for your continued advice, WyomingKnott.

What was not answer on your lan question. The ports all run at the speeds they are rated.

Also you were asking about hooking multiple cables between your router and your switch and stated it broke it when you hooked up 2 cables. That is exactly what is to be expect on devices that do not support spanning tree or link aggregation.

 

[BrianDMG]

You got a good lesson in why you must have spanning tree in a network. Cabling it like you did introduced a loop and broadcast packets will wipe out any device connected to any switch port. Consumer grade equipment many times does not support spanning tree so you immediately get a almost dead network when you cable a loop.

As stated both your switch and router must support port bonding more specifically 802.3ad if you want more than 1g of bandwdith. Still even if you were to accomplish this a single session can not use more than 1 path. You would have to have multiple machines talking too each other on both end to exceed 1g of usage. It doesn't really matter since your wireless can come nowhere near even 1g and you might as well plug machine directly into the large switch rather than plug them into the router and have it forward the traffic to the switch.

I get what you're saying, but I just wanted to point out that wireless doesn't enter into the equation with the question I'm asking. I'm specifically asking about wired connections in my LAN. Also, the only cable connected to the router is the single Cat6 going to the big switch, all other machines are on wired connections via keystone jacks throughout the house (which lead to the patch panel, then to the switch).

Also, thanks again for your continued advice, WyomingKnott.

What was not answer on your lan question. The ports all run at the speeds they are rated.

Also you were asking about hooking multiple cables between your router and your switch and stated it broke it when you hooked up 2 cables. That is exactly what is to be expect on devices that do not support spanning tree or link aggregation.

Right, I appreciate you enlightening me on that - I posted that before I saw your 2nd post that went into more detail (great explanation, btw). So basically, I'd need hardware that could support spanning tree networking in order to have multiple machines all running at ~1Gb each simultaneously?

 

[bill001g]

I am getting somewhat confused because there have been a couple similar posts today and I forget what i post in each.

On you 24port switch lets just assume it is what is called non blocking wire speed since this is very common. Every port can send 1g and receive 1g at the same. So every port on the switch at any instance can actually be running at full rated speed. In theory you could have 48g of traffic active. So in the simple case you have 4 machines. A and B & C and D. you can have files being copied between each pair with no impact on the other pair. In fact A can copy to B and B can copy back to A at the same time...in effect running 2g of traffic. You actually can't do it because of application issues but switches seldom bottleneck a network anymore.

 

[BrianDMG]

I am getting somewhat confused because there have been a couple similar posts today and I forget what i post in each.

On you 24port switch lets just assume it is what is called non blocking wire speed since this is very common. Every port can send 1g and receive 1g at the same. So every port on the switch at any instance can actually be running at full rated speed. In theory you could have 48g of traffic active. So in the simple case you have 4 machines. A and B & C and D. you can have files being copied between each pair with no impact on the other pair. In fact A can copy to B and B can copy back to A at the same time...in effect running 2g of traffic. You actually can't do it because of application issues but switches seldom bottleneck a network anymore.

Ah, that's good to know! Would you say that the single cable connection from the router to the switch would be the bottleneck, then? Or is there no bottleneck in my setup, and I'm just imagining things?

As an aside re: link aggregation: I'm running DD-WRT on my router (a Netgear R7000 Nighthawk), and I was just poking around in the advanced networking sections, and I came across this, which would appear to be able to make ports 3 and 4 a single connection - would this be at all useful in my situation?

 

[bill001g]

dd-wrt does support link aggregation on some routers. Unfortunately your switch does not.

Any traffic between your machines on your switch talk to each other even if you were to turn off your router.

If we ignore the wireless on the router. Traffic would only go to the router if it was heading to the internet. As long as your internet was less than 1g a single cable would be fine. Lets say you had a commercial internet connection with a 10g wan internet. Then the traffic between the switch and the router is only 1g. So if say 2 of your machine each tried to download at 1g/sec from the internet they would be bottlenecked by the 1g connection. If you were to bond 2 connection you could get 2g.

The key thing to remember is traffic between machines on the lan do not need a router. The routers only purpose is to connect to other networks like the internet or the wireless.

 

[BrianDMG]

dd-wrt does support link aggregation on some routers. Unfortunately your switch does not.

Any traffic between your machines on your switch talk to each other even if you were to turn off your router.

If we ignore the wireless on the router. Traffic would only go to the router if it was heading to the internet. As long as your internet was less than 1g a single cable would be fine. Lets say you had a commercial internet connection with a 10g wan internet. Then the traffic between the switch and the router is only 1g. So if say 2 of your machine each tried to download at 1g/sec from the internet they would be bottlenecked by the 1g connection. If you were to bond 2 connection you could get 2g.

The key thing to remember is traffic between machines on the lan do not need a router. The routers only purpose is to connect to other networks like the internet or the wireless.

Thank you! That's exactly what I needed to know. When you say all of the machines would still talk via the switch even if the router was off, do you mean with APIPA?

 

[bill001g]

Well sorta..that is only the DHCP part of the router. Lets say you set the lease very high on the machine or assigned static ip. So all the machine have IP addresses. You can now turn the router off. The machine could still ping each other via IP.

Say we have 2 machines with ip 192.168.10.10 and 192.168.10.11. Now you wanted to copy a file from machine 10 to machine 11. The 2 machines would first send out ARP messages asking for the other mac address. This is all done with broadcast messages so all ports get these and only the proper machine responds. The switch also see this and keeps track of what mac is on what port. So after this point all traffic between the 2 machines only users the mac addresses. The machines for a period of time keep the IP to MAC address mapping and then repeat the ARP process. So they almost don't even need IP addresses to talk. The switch only cares about mac addresses it has no concept of IP.

You will note that this all works even if no router exists. The messages just pass back and forth between the ports on the switch.

 

[BrianDMG]

Well sorta..that is only the DHCP part of the router. Lets say you set the lease very high on the machine or assigned static ip. So all the machine have IP addresses. You can now turn the router off. The machine could still ping each other via IP.

Say we have 2 machines with ip 192.168.10.10 and 192.168.10.11. Now you wanted to copy a file from machine 10 to machine 11. The 2 machines would first send out ARP messages asking for the other mac address. This is all done with broadcast messages so all ports get these and only the proper machine responds. The switch also see this and keeps track of what mac is on what port. So after this point all traffic between the 2 machines only users the mac addresses. The machines for a period of time keep the IP to MAC address mapping and then repeat the ARP process. So they almost don't even need IP addresses to talk. The switch only cares about mac addresses it has no concept of IP.

You will note that this all works even if no router exists. The messages just pass back and forth between the ports on the switch.

Thanks for taking the time to answer my question, you've been very helpful!

 

[awiegman]

Good morning guys... This was a very interesting read and it's given me a lot of insight into my current issue. I don't mean to "thread-hijack" but you guys are all discussing something that is extremely similar to my question and my current setup with one minor difference. I'm running Server 2012 Core with a 4 port NIC adapter. I've been trying to get LACP working and my Cisco Catalyst Express 500 is configured correctly to allow the 4 ports via an ether channel.
However, (I think) my problem is I'm running stock firmware on my Netgear Nighthawk R7000. I have been combing the internet to see if the DD-WRT firmware for my router will let me use LACP.
My virtual machines, when booted with the virtual switch and LACP, it doesn't pull an IP address. I'm thinking that the stock firmware on my router doesn't have the protocol to be able for this to work.

My main question is this. The screen caps that were shown from the DD-WRT screen show you can use LACP on "two" ports on the router itself, but can the DD-WRT firmware send data through a standard port to an external switch that's configured with 4 ports using LACP? I guess the question is, does DD-WRT support the protocol universally or is it ONLY for those two ports?

Like I said, my current setup is very similar...
Router, Netgear NightHawk R7000 (One port going to the switch below)
Switch, Cisco Catalyst Express 500 (4 ports configured with LACP to the server below)
Server, Server Core 2012 running Hyper-V with 2 virtual machines and NIC teaming for both clients with LACP enabled using Hash method.