I understand. That's why I suggested using the ADSL router, which provides the mechanism to automatically configure your LAN clients, via its DHCP server. It doesn’t matter if the ADSL router isn’t needed for internet access, or even if there’s no internet access provided by the ADSL router, it still works.
Your PCs are most likely already configured to use DHCP for configuration purposes. As soon as you plug in the cables from those PCs to the ADSL router, they will self-configure themselves (with the help of the ADSL router's DHCP server). And now all those PCs can communicate w/ each other.
So surprisingly, there's just about nothing else to do except plug in and go! It's all done for you. You don't even need to know as much as I've explained it. It just works. I'm only providing these details because you asked. But again, it's all plug-n-play and automated, as long as you configured it as I described; all the PCs are plugged into the ADSL router.
Sometimes things are far simpler than you might have assumed.
Of course, if you’d like to do it the HARD WAY (manually), I suppose that always an option too.