bjornl :
It is probably not what you did that caused you any grief. There is a lot of nonsense about drivers out there.
Just do some basic PC trouble shooting.
I would suggest that instead you look at autoruns from Sysinternals. Google it. You'll find it on Microsoft's technet pages. Also run MSConfig (built in to OS) to see what it shows you as loading.
See what is running. Are you multi-tasking? Did you get the latest WHQL drivers from nvidia.com? Do you have a virus (Malware Anti-Malware can help you find out).
Do NOT run tools for fools like CCLeaner or Driver Sweeper. Those types of things can ruin PCs (make them unbootable, uninstall core windows components, etc).
I must disagree with some of your statement. Ccleaner is a fine tool to use for those that know that the only part really needed is registry cleaner. This
Will Not remove crucial windows registry entry's. Driver Sweeper with the newer OS's, WIn 8 and newer I believe, will remove some windows components
BUT Driver Fusion, made by the same people, on the other hand will not because it was coded to not remove those components that you are referring to.
@OP, IF you uare worried that the AMD Drivers and software is still on your system then look in Programs and Features and remove any AMD programs like CCC/Radeon Relive and the like. Next go through your drives and remove any AMD files that you find. Just put them in the recycling bin but do not empty it yet. Next use Ccleaner to remove ant registry entrys for the programs and files. DO a restart.
IF you are so inclined to can also do a clean install of the new Nvidia drivers. This would most likely be a practice in futility but will give you peace of mind that it is done correctly.
Now if your CPU is the bottleneck then getting a new GPU will not help in some instances where the CPU is in more demand than the GPU.