That all depends on your chip, and how well the HSF is seated. Some folks say don't OC at all on your stock cooler, but there's nothing magical about an aftermarket HSF. You can probably make 3.7-3.8, or maybe even 4.0 with a golden chip, if you can keep the voltage low enough. I haven't actually touched a 3570k, but I have plenty of experience with the 2500k. A good 2500k could hit 4.0 on the stock HSF, no problem, and still keep safe temps. The 3570k runs hotter, I hear.
But in any event, you'll need to make sure that the fan on the HSF is set so that it will spin up as fast as it will go when things start to heat up, so to keep the chip safe you're going to have a leaf blower on your hands.
disclaimer - any overclocking on stock cooler is at your own risk and not advised
now, many variables come into play, the above mentioned included, as well as case airflow, vrm heatsink air flow, etc etc etc. I would not even think above 4.0 but some claim ok on 4.2/3 - you risk toasting components - is it worth it? Just go get a decent cooler - 30-50 bucks to save hundreds so you dont toast - think about it
^I definitely recommend an aftermarket HSF, this guy here (along with his successor, the EVO) has long been considered the champion of price-to-performance cooling: http://www.newegg.com/Product/Product.aspx?Item=N82E168...
He's $20 shipped after rebate, and you could safely hit 4.2-4.4 without issue (and keep it quiet!) unless your chip is a real dud.
I use rubbing alcohol and cotton swabs/Q-tips. I would definitely take it out of the socket and clean it on its own. When handling the CPU, I put masking tape over the contacts ("pins") and wrap it up over the sides of the PCB so that only the heatspreader is exposed. Then you can go to town.