OCing is all about trial and error. Keep an eye on temps, that's the most important thing. I think 3.8 could be doable, it all depends on your chip. Would I try for more? Maybe! Just watch temps and don't increase voltage much. It might get quite noisy though. I highly recommend a cooler though. That said I got my athlon II x2 from 3.4Ghz to 4.08Ghz with no voltage increases on stock cooler! Best of luck!
Indeed if you love overclocking then for Good safety get an aftermarket cooler, something like Hyper 212+, Stock coolers aren't very good when it's come to overclocking 212+ cooler is very very popular and still very good for overclocking.
Overclocking is very risky task many users blow up thier cpu's due to no care of voltage, First you need to know complete procedure for overclocking then proceed to it, also temps is necessary too.