I have a Belkin Wireless N F5D8233-4 router
http://www.belkin.com/support/prod [...] =F5D8233-4

It has 4 wired/ethernet ports so it should work for up to 4 PC's connected via ethernet, but I cannot for the life of me get any more than 2 to work at the same time. I have it setup like any modem to router setup so everything is set up properly. Right now I have my Playstation 3 connected through wireless, my main computer wired to the router on port #1, my Cable Internet Box in port#2, and finally my downstairs PC is also plugged into port #3 but it doesnt work. Sometimes it will even take out the PC's ability to pick up internet completely, for example right now my downstairs PC won't connect to the net even if I plug it straight into the modem, but my PC isn't the problem, because the router has taken out the cable box AND my upstairs PC in the same manner.

So the issue happens always to the very last PC to be connected via ethernet. Basically if I unplug the Cable box and plug in my downstairs PC it'll work fine, the first 2 connections I plug in are fine, but you can forget about the 3rd and 4th. I've tested everything from different ports, to different cables, different PC's and obviously I've narrowed the problem down to either my router, or my internet company somehow blocking anything more than 2 wired connections.

My questions are, do I need to go through a special process other than just plugging the PC's into the router, they should just work soon as you plug them in right ? the PC picks up the IP and you have net access right ??? but are there other steps I could take to set up these wired ethernet connections manually??
I went through all the settings in the router and none of them specifically have an option to connect other PC's to it.