The trick is not to be nervous. If you are not confident then make a note of how many connections are made with your old power supply and match it up with the one you have installed. That way you will know if you have missed one.
The worst part that I find is the 24-pin motherboard connection as it is usually very fiddly and hard to push in. Just be firm but do not crack the motherboard.
I really would not worry, there is not much that could go wrong with a PSU switch.
Random thought of the day:
I feel like you're the kind of person who would like a Modular PSU. Makes cable management (and basically everything else for that matter) a lot easier and neater. Recommend the OCZ ModXStream Pro; 50w less powerful, though... you give it up for the sexy-factor...
Sorry about that. You'll have to excuse my obsession with Modular PSUs...
Anyway, just to back up the crowd even more, yeah, it's really that easy. Not much else to be said. The really tricky part comes in cable management, which may just be on the edge of nonexistent in an HP system. I'm sure you can handle it though. Words of reinforcement, those are.