What is your current GPU? The performance hit is very dependent on the game you're playing, something like Fallout or the Witcher 3 you will notice a bigger impact on performance. I run a i3-4160 with a 1060 and generally my FPS is very steady on high graphics settings. I would imagine on most games you'd be able to get close to 60fps @ 1080 with some tweaking.
I upgraded from a 960 to a 1060 and the difference was very noticeable (The Division for example was 20+ fps faster) so clearly my CPU had headroom in games to make use of the extra power.
I find that I sometimes have to dial back on CPU intensive settings to maintain a solid 60fps, while with something like Overwatch I can pretty much max the graphics settings and the performance is rock solid.