I am having the same problems. I am running Windows 7 64 bit. I have no other problems with my USB headset with the controller unplugged.
It may be that the controller is trying to use it's own drivers for an attached headset. Maybe something with the Microsoft controller drivers.