With an ATX power supply you cannot easily turn on the P/S without an attached MB and the power button sub-board. ATX power supplies are always 'on', in that they send low current 5V to the MB at all times unless unplugged from the wall. The MB has a controller that sends power up and power down signals to the P/S when you push the button.
If you want a cheap 12/5/3V power source for hobbyist purposes dig around for an old AT power supply. It would have to be pre-Pentium 2 or about 1997, many early pentium models used AT P/S's, all the way back to PC AT's in 1984.