Loud Buzzing Noise When Turning On Computer, 1 year old PSU suspected

Had PSU for about a year. It's a Silverstone strider 560 Watt http://www.newegg.ca/Product/Product.aspx?Item=N82E16817163112.

Everytime I turn on my computer it makes a very loud and noticeable sound, which slowly dissapates.

My Rig: -AsRock 939 Dual-SATA Motherboard
-AMD64 3200+ Oreleans CPU
-1GB Generic RAM
-ASUS 1950 Pro 256MB Video Card
-Creative Soundblaster Audigy SE Sound Card
-Maxtor 250 GB Hard Drive

The possible fans that can create this noise are:
-The standard CPU fan
-The Standard video card fan
-A large front fan that came with my generic case
-A 80mm fan on the back of my case
-The 120mm fan in my PSU

The sound seems to be stonger in the back, although it could be just me.

The reason I suspect it's the PSU is because I had a similar case with an older computer a while back. I still had the warranty at my local store so a had it replaced, and the noise dissapeared.

Currently have the same sound, although milder, on an old Dell Computer.

I'm just doubting myself as I thought Silverstone is a good quality company.

Could this be the PSU?
If not, what else could act the same way?
Just bestranged that the sound goes away after awhile.
Never returned a defective product using a warranty before. Advice?

Thanks in advance!