I prefer to pull out network cards that I'm not using to save the computer's resources for the functional parts and processes.
First, I'd go into Control Panel / System / Device Manager and "uninstall" the old network card, along with its drivers.
Then, shut down the computer, physically remove it.
If you've got a USB plug-in card, then you can power up the computer. Some of those say "Install drivers first, THEN plug in the USB card..." and others will merely ask for the Drivers after it detects the USB device.
This will save Windows from being confused about loading drivers for equipment that may or may not be functional.