It needs to be applies anywhere that a heatsink or cooling surface needs tight contact to a heat generating surface, like a GPU and CPU, mostly you only need to worry about a CPU.
It will need to be fully cleaned of the cooler and/or CPU after cooler removal. Many aftermarket coolers come with it pre-applied though.
Unless a GPU is getting abnormally high temps it is not advised to remove the cooler and re-apply thermal past as the manufacturer has already done so.