When I was a total noob I had removed my CPU heatsink once, and didn't reapply thermal paste(did something like that exist?) and put it back as is. There was no drastic change in temperatures.
The case may be different for other CPUs as mine is fairly old and is only 65W.
To be safe, always reapply thermal compound.