Are your computers using DHCP or static IP?
What is the IP of the router?
What is the result of "Ipconfig" on a command prompt of your PC?
(What is the gateway address being displayed?)
Does the router have DHCP enabled?
Does the router have MAC Address filtering enabled?