PS... The reason this is occurring is because the BIOS isn't disabling the onboard card, it is just using the new one over the old one as default. XP knows nothing about this and starts the video driver like nothing has occurred. The new card's video driver is incompatible with the "current" one and can't run at the same time. If you want you can install all of the drivers for the new card before you disable the old one. No need really to use safe mode as long as you can see what you are doing. Game On!
BTW that "blue plug" is analog VGA and you should avoid it like the plague these days
Use DVI whenever, wherever possible and HDMI as a last resort.