It is that easy and then again it isn't. Is the psu you have now bottom or top mounted? Is it modular? The majority of the newer power supplies are bottom mounted and intake cool air from the bottom of your machine and exhaust the hot air out of the back. Older power supplies took hot air from inside the machine and exhausted the air out of the back. Needless to say the bottom mounted psu's are more cooling efficient.
Which PSU are you looking at as a replacement? What type case do you have? ATX or M-ATX? And does your case have room to accommodate the new PSU? Measure first. Also consider getting a modular psu because you can prevent a lot of clutter in you machine by eliminating cables you don't need. Then it's just a matter of connecting the cables properly; 20 or 24 pin main power cable to motherboard, 4 or 8 pin cable to cpu and 4 pins to sata or molex for optical drives and HHD.
Take your time. The first time I did it was on a pre-built HP and it took about an hour and most of that time was spent trying to manage all the cables I didn't have to use.