Not with a splitter. You need to use a switch or you can connect one router to the modem and the second router to the first one. How many IP addresses are provided through the modem? If only one, then using a switch won't work.
GhislainG is right, most ISPs will only issue you one IP address, so you usually cannot simply plug two routers into a switch plugged into your modem. What is it that you are hoping to achieve by having two routers? Knowing this will help us to suggest the best way to configure them.