Use a soft cloth of some sort, and isopropyl rubbing alcohol. I usually use a low lint rag, and the 91% isopropyl rubbing alcohol. I learned the lesson a long time ago about not using a card or anything hard to clean the thermal pad off of the bottom of heatsinks. Especially when it comes to the softer copper based ones.
I ended up having to find a way to smooth out the base on a $50 all copper hsf unit back when I put my old Athlon XP rig together. Back then all copper hsf units weren't all that common or as cheap as they are now.