See the literature which came with your router for how to access its user setup screens. You'll need to note the default IP address of the router (usually in the range 192.168.0.1 though it varies by model). Connect the computer to the router with an ethernet cable. Open Internet Explorer and enter the router's IP address into browser's address bar as though it was a website.
You should then reach the login page to the router's setup screens. The User name will probably be "admin" (without the quote marks) but the password can vary according to the brand of router .