I have seen this with both overpowered(850 watt on a 150 watts of computer) and underpowered units(450 watt units @ 300+ watts).
What is the power supply and system specs you have.
I have seen this, but it seems to be an interaction with parts. Not always bad(no damage), but clearly you do not want to have to hear that sound all the time.
Some power supplies have an extra high uF rating capacitor on the PCI-E/CPU power lines that are supposed to reduce or even eliminate this noise.
Switching power supplies are inherently noisy, but have high efficiency. With the right filtering they should be good for audio, but things happen on a board with so many parts needing power in differing amounts constantly.
I can not give you a yes or no for a certain power supply(but an interested in you current power supply.), because it is an interaction of all parts. Most users just hear the coils on the card or board whining, but some do hear it on the speakers/headphones.