You can assign IPs 1 of two ways:
Assign them at each individual computer. You will go to the network adapter settings, and under IPv4 you will hit properties and then you can set the IP address. Your router has a function to provide addresses using what is called a DHCP server, you can go into the config page and turn this off, or edit the dhcp server address range to a smaller number and make your PCs an IP that is not in that range. So say your router is IP 192.168.1.1 and your DHCP server range is now 192.168.1.150-192.168.1.254 then you would set your PCs with IPs that are between 192.168.2-192.168.149.
The other way is doing it at the router, not all routers have this feature (now all upgraded router firmwares like DD-WRT and tomato do. Each router/firmware has a different way of doing but from in the router config page you can associate an IP address to a specific MAC address and then your router will assign it that IP everytime. In this case you would assign the IP to an address that IS INSIDE the DHCP server range (vs the first method where you use an address outisde the server range).
I use this method at home because I want static IPs but dont want to have to change settings on mobile devices everytime I want to use a different wifi connection when I am not at home.