The stock heatsink comes with thermal paste on it. It is kind of tricky, but the best way to tell is to turn the heatsink over and look at it. Sorry, just had to be a smart ass.
You can use the heatsink just as is with the stock thermal paste, or if you want to, you can take a little alcohol and clean it off, then apply your own brand of thermal paste. If you are not going to do any big overclocking, and you are careful to mount the heatsink properly, the paste that comes on it from the factory will work just fine.
If you ever remove the heatsink after you have installed it though, you will need to clean the heatsink, the processor, and apply new thermal paste of your own.