Did you uninstall the old drivers before you installed the new graphics card?
Usually its good practice to uninstall drivers -> swap cards -> install new drivers.
Try uninstalling the drivers completely and restarting, it should still boot (although your display will be a bit average). Check if you have issues shutting down with no drivers and then see if when the drivers are installed the issues come back/remain.
Regardless, it sounds like a software issue if you are able to boot.
Might be worth doing a fresh install of Windows if all else fails.
Its probably worth checking that all your BIOS settings are correct or at least set to their default values (or optimised defaults whatever). Doesn't sound like this is the cause of the issue however.