Yes, quite a bit, but you'll still have a better experience moving to a faster card.
Your CPU of ultimately what determines what the highest framerate you can get is. With a new GPU, you can run higher graphical settings but you'll never exceed the framerate your CPU is capable of.
For what it's worth, Nvidia cards are easier on the CPU in DX11 games, which 99% are right now, and thus perform better when paired with a weak CPU. You may be better off with a 1060 3GB or even 1050 Ti.