Yeah, all you have to do is touch the two pins and it will instantly spark to life (sometimes literally).
The power switch on a computer is a momentary-on switch which means that while you are pressing it, the circuit is closed (it allows electricity to flow through). It isn't like a light switch; it isn't connected between the power supply and the motherboard, directly allowing or preventing the flow of power between the two. It operates more like a signal to the power supply that you want to start the computer, to start delivering power to it.