I have this issue which I believe is ground related, but not too experienced with this for audio.
It happens more if I using regular headphones with a 3.55mm jack. I also use a usb microsoft lifechat and it is less noticeable, but still present.
It comes and goes, it seems to be related to the monitor initially. The more of the color white, the more it buzzes. Scrolling with the mouse wheel seems to generate the most. Also on the desktop, if left click draggin to make a big select window, it will also do it.
I have noticed it if I blast my speakers to the max, so I don't notice it otherwise.
And for good measure it still makes these sounds if the monitor is off.
I initially got the usb headset because of how bad it was with regular headphones.
I have a different headset now, same Sony model, just new, and it isn't as bad.
Once i have a movie or game going, it isn't noticeable. it's those times between that its awkward.
For reference my sound is onboard Asus P8P67 Pro 3.0
For those who are still looking for solution, i can provide one not guaranteed it will work for you, but for me it got the job done. Not so long ago i bought new headphones and yesterday i noticed that whenever I use scroll or press keyboard there is quiet rumbling noise. Ar first i was like meh, but I use scroll quite often so i couldn't ignore this humming. First two links from google didn't help me, so i thought "It's headphones, so let's have a look at sound settings" Long story short, go to Master Volume, click Advanced and uncheck Microphone boost. Worked for me. Please note: there was noise only when scrolling mouse wheel or pressing keyboard. Also, for the reference, my headphones, mouse , keyboard are wired. Integrated sound card. ASrock motherboard.