Yes you will need to clean off the old goop. With the paste that comes on the stock heatsink this isn't such a fun task.
Clean it off using 90%+ alcohol (Such as Isopropyl) and personally, I like using Q-Tips.
Unless you plan on using the stock cooler again, you probably don't need to do this to it too. Just stash it somewhere or throw it away.