GPU is easy, PSU requires a little bit more work.
GPU is just one screw, pull out old card, put in new one, screw it back in.
PSU requires 4 screws and disconnecting a few power cords, and some possible wiggling it around to get it out.
When replacing a GPU, you also have to make sure you press the release tab at the bottom back end of the card so that you don't wreck your motherboard. You also usually have to plug in one or two 6-pin PCIe power cords from your power supply into the new card.