Yes just set the second router to run as a AP. You want to run on a different radio channel and even a different SSID if you want.
Most newer routers have running as a AP as a feature. You can buy a real AP but they tend to cost more since they are designed to run PoE. But you can make almost any router be a AP by connecting the cable to the lan port and disabling DHCP on the second router.