It wouldn't be the HDD because the machine doesn't start up at all.
Op, check the board whether the 'power light' is lit or not. If it IS lit, I strongly doubt it'd be the PSU. Open the PC and try to identify where the noise is coming from. If you can't identify where, the next thing to do would be to run the pc barebones (plug in only mobo, cpu, ram, psu) and see if it fires up. The process of elimination. If it fires up, I'd put my money on a shot GPU. If it doesn't - I'd say you most likely have a shot mobo.