It depends somewhat on the games you play. MMOs (WoW, Guild Wars 2, Tera, Planetside 2, DayZ, etc), you'd actually see a bigger improvement switching to an Intel CPU than upgrading your graphics card.
Most other games, the graphics card will offer a much better upgrade.