It can do. If you had a finely lapped and well mated cooler it might not matter, so long as no air pockets are trapped in the interface. If you have crusty old TIM and you pop the heatsink off/on then you can expect to trap some air in there, so yes. It can.
When I was a total noob I had removed my CPU heatsink once, and didn't reapply thermal paste(did something like that exist?) and put it back as is. There was no drastic change in temperatures.
The case may be different for other CPUs as mine is fairly old and is only 65W.
To be safe, always reapply thermal compound.