There's a little rubber thing on the other end of the pecil called an Erasor. This has been standard equipement on both traditional wooden pencils and inexpensive mechanical pencils for quite some time. There is no need to modify a pencil that is not so equiped, as a package of them usually cost less than $1 at your local grociery store. Look in the sationary dept. I prefer Stanford pencils, both for graphite quality and rubber quality of the erasor. Less expensive pencils often include an erasor that smears.
Oh, and if your really fussy, you can rub off any residue with alcohol.
Cast not thine pearls before the swine