For today's games and at higher resolutions both those components are bottlenecks. If you're playing at low resolutions like 1360x768 and still getting bad FPS then it's definitely a CPU bottleneck.
My 670 played any modern game perfectly smooth on max settings, (1360x768) but when I went to 2560x1080 and beyond the card really struggled. If you want to play at high res, then the 660 is definitely the bottleneck, and your CPU to a lesser extent.