Desktop switch kills routing

Archived from groups: comp.dcom.lans.ethernet (More info?)

nntp chip wrote:

> I run a network where three different lan:s are used. Between the
> buildings at every site the traffic flows through tagged ports in layer-2
> switches. (ASCII-art and switchmodels below) When traffic need to go
> somewhere outside that site a layer-3 switch routes it onto a carrier
> network kept separated from the three other vlan:s.

> Enabled spanning-tree on all switches to kill off nasty loops.

> So far so good.

> Then some student connected a simple desktop-switch and made a loop within
> that little switch. Somehow the spanning tree did not work correctly in
> that situation. The entire student-vlan stopped dead. While searching for
> what was going on, the administration people started complaining too; They
> could reach the local servers, but remote servers and internet was
> unreachable.

If the loop is within a non-STP switch there isn't much
the others can do about it.

> Set up lab to study things a little closer.

> Found out that when one of the vlan:s was looping, the other vlan:s worked
> within that site, but routing soon stopped in the layer-3 switch. The very
> second i disconnected the offending desktop-switch everything went back to
> normal again.

If vlan-110 is allowed out through the router, it could easily
saturate the router. A loop creates many copies of a packet, and
they can all get sent out through the router.

> Any ideas how to stop this from happening and keep the routing going? The
> admin-network Must Always Be Reachable, so I dont like the idea that some
> lousy desktop-switch can wreak such havoc...

(snip of diagram, I believe it is explained well enough.)

If your equipment allow, limit the bandwidth allowed for
each vlan into the router. That at least gives you a chance,
as it might be that you are just saturating the uplink.

If you are saturating the router itself then it is harder.

Well, limiting on the vlans through the layer 2 switch would
help a lot. Not quite as much if you can only limit at the
router.

-- glen

Archived from groups: comp.dcom.lans.ethernet (More info?)

"Steve Horsley" <shoot@the.moon> wrote in message
news:coj6pc$4n3$1@news.freedom2surf.net...
> nntp chip wrote:
>
> <snip long and thorough description>
>
> My first guess is that the student VLAN was completely swamped
> with a broadcast or multicast packet being repeated at full wire
> speed by the rogue switch (effectively a traffic generator). And
> I guess that this broadcast flood drove the router (switch 1)
> into total CPU overload where it couldn't function properly.

Agreed - i have seen this happen before as well.

it may be the broadcast level generally, or just IP broadcasts and
multicasts depending on your layer 3 kit and the traffic profile in the
network.
>
> It's difficult to know what to do about that, other than
> perhaps using a separate router / L3 switch to route between
> the non-admin LANS and the carrier network, just to separate
> and protect the admin router. Perhaps with rate limiting.

Some layer 2 switches support broadcast rate limiting. The idea here is to
limit the number of broadcasts propagated to the rest of the network to
limit the spread of this kind of fault.

This does work, but probably isnt turned on by default if available on your
hardware, or the level is set so high that the limited stream still chokes
the layer 3 switch - experimenting may be the only way to choose a
compromise setting.

Alternatively, some layer 3 switches support control plane limiting,
intended to limit this kind of denial of service attack

These are the kind of "bells and whistles" features that isnt universal, so
may not be useful for your current kit.
>
> Steve
--
Regards

Stephen Hope - return address needs fewer xxs

Archived from groups: comp.dcom.lans.ethernet (More info?)

"nntp chip" <me@privacy.net> wrote in message
newsan.2004.12.02.17.07.35.646781@privacy.net...
> On Wed, 01 Dec 2004 20:14:31 +0000, stephen wrote:
>
>
> > "Steve Horsley" <shoot@the.moon> wrote in message
> > news:coj6pc$4n3$1@news.freedom2surf.net...
> >> nntp chip wrote:
> >>
> >> <snip long and thorough description>
> >>
> >> My first guess is that the student VLAN was completely swamped with a
> >> broadcast or multicast packet being repeated at full wire speed by the
> >> rogue switch (effectively a traffic generator). And I guess that this
> >> broadcast flood drove the router (switch 1) into total CPU overload
> >> where it couldn't function properly.
> >
> > Agreed - i have seen this happen before as well.
> >
> > it may be the broadcast level generally, or just IP broadcasts and
> > multicasts depending on your layer 3 kit and the traffic profile in the
> > network.
> >>
> >> It's difficult to know what to do about that, other than perhaps using
> >> a separate router / L3 switch to route between the non-admin LANS and
> >> the carrier network, just to separate and protect the admin router.
> >> Perhaps with rate limiting.
> >
> > Some layer 2 switches support broadcast rate limiting. The idea here is
> > to limit the number of broadcasts propagated to the rest of the network
> > to limit the spread of this kind of fault.
> >
> > This does work, but probably isnt turned on by default if available on
> > your hardware, or the level is set so high that the limited stream still
> > chokes the layer 3 switch - experimenting may be the only way to choose
> > a compromise setting.
> >
>
> Broadcast rate limiting exists, but even when set to the lowest possible
> limit it doesn't seem to work as intended. Looks like the excessive
> traffic is not classified as broadcast and thus never triggers the
> limiting behaviour.
>
>
>
> I'll try experimenting with rate limiting for those locations where
> bandwith is not an issue.
>
> Still some places require huge bandwidh, so another router or layer
> 3-switch should do the trick there.
>
>
> Thanks for the ideas.
>
>
> > Alternatively, some layer 3 switches support control plane limiting,
> > intended to limit this kind of denial of service attack
> >
>
>
> Do you know any brands/models that use control plane limiting? Might be
> nice to read up on that.

cisco have put this on a lot of their recent stackables, as well as the
chassis switches - they also support broadcast and multicast storm control.

here is a link to the Catalyst 3550 with some info in the feature list
(layer 3 enet stackable)
http://www.cisco.com/en/US/product [...] 913d0.html
>
>
>
>
> > These are the kind of "bells and whistles" features that isnt universal,
> > so may not be useful for your current kit.
> >>
> >> Steve
--
Regards

Stephen Hope - return address needs fewer xxs