I'm confused - you turned on the PC but it wasn't hooked up to the monitor? If you turned it on and everything was hooked up, then your CPU was definitely running. Unless you just mean that you turned on the PSU switch but did not actually power on the computer?
If that's what happened, then it's definitely the PSU, because nothing else should have been getting power. If you actually powered on the whole computer, then it could be any part.
Smell the PSU. Does it have a burned smell to it?