Yes, if you can't see by removing the other side you you are going to have to pull the motherboard. I'd be sure to ground yourself before you start. You might want to start the computer and let it heat up a little then shut off to make taking the heatsink off the CPU easier.
I use a Q-tip with a little rubbing Alcohol (Isopropyl Alcohol) on it to clean off the old thermal grease. Be sure you remove all of it off the CPU and the heatsink. Then be sure it is dry before applying the new thermal grease. Some use the Pea method I just put a very thin film I can see thru on the CPU heat spreader then put on the fan heatsink.