in simple terms, when you move around you can accumulate an electrical charge. when you touch something conductive like a metal case or circuit board, the charge dissipates, which essentially means that electrons are flowing from you through the object that you have touched. if you touch a very tiny wire on a circuit board, you can send a large number of electrons through it in a very short time period. too many, in fact, and it will destroy the wire, and oftentimes your component. by touching the metal case you allow the electrons to dissipate throughout the large volume of metal, instead of a tiny wire on an expensive component.
Yes it does require to PSU to be installed and plugged in, not turned on. Also you don't have to have the PSU's cables plugged into the individual components.
The 3rd prong on the PSU power plug is the ground cable, and when that plugs into the PSU it grounds the PSU. When the PSU is installed in the CPU case, the screws going through the case and PSU ground the metal of the case.