Paper towels may leave residue, coffee filters do not and work fine as I have done it many times,and yes remove the cpu and do it without the cpu in the socket, because you do not want to spill any alcohol into the motherboard.
Paper towels are fine, I've used them for years with no ill effects.
Also, FYI, alcohol wouldn't hurt your motherboard. Rubbing alcohol is nonconductive and evaporates very rapidly. It's still a good idea to remove the CPU for cleaning, however. I like to get as much of the thermal paste off as possible before unclamping the chip, then clean it again more thoroughly before putting it back.