Sign in with
Sign up | Sign in
Your question

Dual network question - 100Mb Internet & Gigabit private

Last response: in Networking
Share
August 6, 2009 6:13:47 AM

Greetings.

I have what might be a rather strange question, but please bear with me.

I am planning to build a dual-network setup. I have a number of workstations and a mess of servers, and I want them to be able to communicate with each other on a "private" Gigabit network, and with the Internet on a "public" 10/100 network (since local ISP's tops out at a theoretical 14Mb/s, a 100Mbps network is quite adequate).

One of the reasons for this is that I want my servers "segregated" on their own Internet-facing connection, but fully connected with my workstations. That way, the Internet connection for my servers can have its own dual gateway setup, making use of three OpenBSD gateway/dns servers for load balancing between a dedicated cable modem and ADSL modem. My workstations will be on their own ADSL modem so that their traffic *cough*bittorrent*cough* doesn't foul my server's bandwidth.

My main question is how I engineer this so that any *internet* request that any computer makes is preferentially sent out via the 10/100 network, but that any *network* resource request (such as access to my 10TB Raid6 NAS) makes use of the private gigabit network. I have some rough ideas, but I really don't want to re-invent the wheel.

The key point is that I want to make any and all communication strictly segregated, preferably O/S agnostic and implemented on the network itself. TCP/IP, POP3, SMTP, IMAP and BitTorrent/Gnutella traffic goes out exclusively via the 10/100 network, and FTP/private networked drive sharing goes via the Gigabit network.

As well, I would like to have the 10/100 network machine-segregated as well. That is, the only thing visible on the 10/100 network to any one machine should be the gateway. In order to see another machine (even if it is on the same 10/100 network) the Gigabit network would need to be used. Other machines should not even be pingable on the 10/100 network. To all intents, a machine on the 10/100 network should not exist to any other computer on that network, no matter what the protocol.

Suggestions?
August 9, 2009 5:16:16 AM

Thats a good question, I'd like to hear the results, What switch do you have? or are you still waiting to purchase the hardware?

If you have a good managed gigabit switch I doubt there would be much harm in combining both networks into 1. I understand you want optimal peformance out of your local network, but, I seriously don't think its worth the extra effort of trying to segregate it. If anything invest in a switch that can team two gigabit ports together for your servers, so you can alleviate any bottlenecking at the server when two or more pc's are requesting files from it.
August 9, 2009 6:48:52 PM

Well, all the computers already have a single 10/100 port (usually onboard) and a single gigabit port (most of them with a 3Com PCI-X card, otherwise a second onboard port).

Besides, my main objective is twofold:

First, with any Internet-facing or Internet-orientated communication, not only do I want it to go over the less efficient 10/100 network, but I also want to “isolate” each computer (by any method whatsoever) so it thinks it is the only computer on the network. That way, it isn’t tempted to use the 10/100 network to communicate with any other server or workstation.

Second, I want all internal communication to go over a pure gigabit network. Three machines will have RAID-5 arrays (but only one will be a "permanent" data store), so the ability to swap gigabytes of data very quickly between any two machines is critical.

Since my workstations will be running data-hungry protocols over the Internet (BitTorrent, etc.), I want them on a completely separate ADSL modem or Cable modem (probably the former), so that their bandwidth does not impact my servers (which will have a pair of load-balanced connections - one ADSL and one Cable Modem). Hence, the two separate Internet-facing 10/100 networks (one for workstations, the other for servers, so the workstation traffic does not impact the ability for the servers to service site visitors), and hence the need for a private gigabit network (so workstations and servers can communicate directly when moving data around).

The real question is how to do this. Ergo my original question.
Related resources
August 16, 2009 7:09:57 AM

Bump
August 25, 2009 9:22:16 AM

Bump
August 26, 2009 5:14:45 PM

I guess no-one has an answer. I thought that one solution might be a higher-end switch that could do packet filtering; one switch would allow only Internet-bound TCP/IP traffic, and only to/from the gateway (computers on that switch wouldn't even be able to see each other, much less communicate), and the other would allow everything except Internet-bound TCP/IP traffic (so that computers could see each other and share files without impacting Internet-bound traffic).

Remember, I am trying to do this in an OS-agnostic manner. If I were wanting to do this on the O/S level, and I had only Windows machines, it could be quite simple: have TCP/IP on one NIC, with all NetBIOS over TCP/IP shut down, and have only NetBUI on the other NIC (no TCP/IP v4 or v6). That way, any box-to-box file and printer sharing would go over NetBUI, and anything Internet-bound would make use of TCP/IP, ensuring that the demands of each network would be highly regimented.

Unfortunately, I have only one Windows workstation and one Windows server left. Everything else is either MacOSX, BSD, Linux, Solaris or BeOS (Haiku). Unless I can split protocols between NIC's on other OS's like I can on Windows (multi-platform NetBUI, anyone?), I require a hardware-based solution.

So, anyone?
August 27, 2009 12:31:43 AM

If all machines have dual-NIC, I think you could achieve the "2 different internet connections" by creating 3 subnets.

The first one would connect all your computers without any internet connection with their 2nd NIC (might be important because of PRIMARY network assignation depending on OS).

The second one would connect connect all the workstations 1st NIC and connect to the internet.

The third one would be the same, but for servers.
August 27, 2009 3:09:12 AM

Zenthar said:
If all machines have dual-NIC, I think you could achieve the "2 different internet connections" by creating 3 subnets.

The first one would connect all your computers without any internet connection with their 2nd NIC (might be important because of PRIMARY network assignation depending on OS).

The second one would connect connect all the workstations 1st NIC and connect to the internet.

The third one would be the same, but for servers.


Hmmm… In some cases, as with BSD and MacOS, there are no distinctions between network cards. Even in Windows, a request for a web page would normally go over any network card which is connected and has a gateway IP programmed into it. It is without that gateway IP address that a card would refuse to pass along an Internet-bound request. Problem is, I am not sure if this is limited to Windows or not, so I am not sure if this would work.

Remember, I need this entire Rube Goldberg contraption to be OS-agnostic. That is, it can't depend on Windows or BSD or MacOSX. It must function exactly as required regardless of what OS is running on a box.

On top of it, I'm really not sure this can be reliably done via software/OS. I have three basic requirements:

1.) Any service requests (web pages, eMail delivery) that come from the Internet to the servers must have access to 100% of the bandwidth of the Internet connection (and associated internal network) 100% of the time, and with 100% reliability.

2.) Any BitTorrent/Gnutella communication to or from the Workstations must have access to 100% of the bandwidth of the Internet connection (and associated internal network) 100% of the time, and with 100% reliability.

3.) Any file and printer sharing between servers and workstations cannot go over the Internet (obviously), or over the same networks as Internet-bound communication, as it would violate rule #1 and #2.

Since I will have both ADSL and Cable connections, rule #1 can have one Internet connection, and rule #2 can have the the other. That way, they can both fulfil their 100% mandates without stepping on each other's metaphorical feet. Problem is, they are then completely isolated from each other. Ergo, the Gigabit connection for moving files back and forth between servers and workstations. And since the Gigabit network is for the machines to communicate with each other, I want them to be totally isolated from each other on their respective Internet-facing connections. That is, they think that they are the only machine hooked up to the gateway, so as to remove any chance that they might "foul up" the Internet-facing networks with any machine-to-machine communication.

I strongly believe that there is a hardware-based and OS-independent way of doing this. Maybe it needs an expensive low-level Switch, or maybe an OpenBSD machine acting as a switch, with 3 or 4 4-port NICs. I don't know. That's why I am asking here.
August 27, 2009 10:55:07 AM

The gateway explanation makes sense. An alternative would be to use a switch, but then you would need to configure every device's IP manually. If you could find a router that does not "register itself" as the gateway when DHCP is enabled, then perhaps you could save yourself the IP configuration. A way to do that might be to get a router that would support WRT-DD and customize it as you wish. Might require some time, but might save you the money of a professional switch.
August 27, 2009 3:05:01 PM

Zenthar said:
The gateway explanation makes sense. An alternative would be to use a switch, but then you would need to configure every device's IP manually. If you could find a router that does not "register itself" as the gateway when DHCP is enabled, then perhaps you could save yourself the IP configuration. A way to do that might be to get a router that would support WRT-DD and customize it as you wish. Might require some time, but might save you the money of a professional switch.


The only downside to "the gateway explanation" is that I cannot use my usual range of 192.168.0.1 and up as IP addresses across both the Internet-facing and the Gigabit networks. My only choice is either to use the private group further up or to split the 192.168 group by using a custom subnet mask. At least then I can set up a DNS server on the Gigabit network and apply custom non-Internet names to each machine (so I don't always have to call them by IP address).

In the case of the Gateway address, it may be that a network card cannot function properly without some sort of a Gateway address. Would the loopback address (127.0.0.1) be an appropriate value (as a Gateway address) to ensure that Internet-facing traffic does not travel over that network? Or will all zeros work for most operating systems?

PS, I intend to hard-code all of my IP's. I know all my machines, and there is no wireless, so there is no need to use DHCP.
August 27, 2009 10:42:16 PM

Ok, so I think I understand what you are trying to do but I am not certain, tell me if this is correct:

1. You want to have 2 separate networks, 1 for servers and 1 for workstations.
2. You have 2 internet circuits, you want servers to use one circuit and workstations to use the other.
3. You want the 2 networks to be able to talk to eachother across your lan.

If this is incorrect let me know. If this is what you are trying to do I think you are making things too complicated; you won't need multiple NICs on each server, you just need to use a few routing and NAT tricks. There are a number of ways to skin this cat depending on what network hardware you have or are willing to buy. Do you have any managed switches? What type of router(s) are you using? Can you provide a picture of what you are trying to do?
August 27, 2009 11:05:54 PM

The thing is he also doesn't want the internet traffic to "interfere" with the internal network's traffic, that is why the 2nd NIC comes in handy.
August 27, 2009 11:19:05 PM

Zenthar said:
The thing is he also doesn't want the internet traffic to "interfere" with the internal network's traffic, that is why the 2nd NIC comes in handy.


Exactly. And since most of my boxes already have *only* 10/100 NICs built in, I installed Gigabit NICs for the private network. The 10/100s are fine for any Internet-facing communication, since most SOHO and small business connections only have a theoretical limit of around 14Mbps anyhow.
September 20, 2009 8:57:37 PM

I just ran across a reference to Virtual Lan's in the Cisco CCNP books. Would OpenBSD gateway have the capability to create virtual lan's on demand? That would solve the "machine isolation" in the Internet-facing 10/100 connections, and prevent them from seeing each other.

So, is this possible, and how would I best implement it? I've tried Googling it, but info out there is surprisingly thin. Remember, I will be having an OpenBSD machine act as a gateway, with dumb switches hooking in all the other machines that require Internet access.
!