if you really hate your processor that much feel free to just stick it in the oven and watch it melt... you need aftermarket cooling to do any real OCing.
However with that small of an OC only 300 mzh it wouldn't take a massive cooler to get the job done. I think you might could even just raise the multi to get it to 3.9 ghz, and possibly bump vcore a tiny bit if it won't boot.
The FX 4000 series chips should all be able to hit 4ghz on stock cooling. They are not that power hungry nor do they run particularly hot. You might be able to do 3.9 by just turning up the multiplier.
You can easily hit 3.9 with the stock cooler. I would say about 4-4.1 is about the limit since youll most likely have to raise the vcore a hair creating more heat
Not even going to ask about his current temps? braver than I!
Ive seen a number of posts on the quad cores they dont get nearly as hot as the 8 cores on the cooler they provide. You can get decent overclocks on the stock cooler but i still wouldnt use the stock cooler in the long run.
As long as theres and intake and exhaust fan it cant be to bad