If you haven't tried already, start with an uninstall of your current nvidia drivers
Then download the latest version directly from nvidia and install them
If that doesn't improve things, check task manager. Is anything hogging CPU/RAM usage when you're not gaming?
Also, use msi afterburner and rivatuner
Use the onscreen display to monitor CPU, GPU & RAM usage and also CPU & GPU temps and see what results you get
What resolution & hz are you displaying at?