How to wipe thermal paste off GPU?

Use alchohol with some q-tips. Avoid any scratchy-objects (fingernails, plastic cards, etc).

Seems the manufacturer likely used a thermal epoxy or adhesive rather than a grease. What you could try is an epoxy remover, such as acetone, or perhaps even something stronger [like DMC (Dimethylene Chloride) or DMF (Dimethylene Formadihide)], but all these chemicals can eat through other parts of the card.

The only sure fire way to get it off without chemical risk to the rest of the card is to chip and pick at it. Doing so will likely leave the GPU heatspreader scratched, but if done carefully, those scratches will be minor and easily filled by a quality thermal grease, which should not hinder it's ability to transfer heat to the cooler too drastically.