You should not need to disable the device in
Device Manager, but there would be no harm in disabling it either. Simply find the device in Device Manager, right click, and select Disable. If the device has indeed failed and will remain disabled,
there is no reason for any related programs to remain in Programs and Features. Both Device Manager and Control Panel can be accessed in Windows 8 from the menu of administrative tools which appears when you right click in the bottom left corner, where the Start Screen would be launched.
The audio device embedded in the motherboard may also be able to be disabled from the system firmware, though it cannot on the Dell Inspiron 1564 test machine which I have on my test bench.
The important setting to ensure that sound functions as you intend it is to ensure that the new sound device is considered default in the Sounds control panel. You can access this by right clicking on the speaker icon found in the System Tray and selecting Playback Devices. Select the USB device and select the Set Default button. You may also wish to do this for the Recording tab if you use a microphone.