I have the same cpu and MB. At first I just had the stock cooler. I didn't spend much time OC'ing at all but I tried using the easy tune software that came with our MB and would get instability with the stock cooler at about the same speed (3.8ghz) . Again, I wouldn't even call my effort a real try but I think that might be the upper limit of the stock cooler. If I were more experienced with overclocking I could probably get a stable 3.8 on that cooler. I bought a Thermalright U120 and the load temps went down 22 to 24 C. I haven't tried OC'ing with the new cooler though I am sure I could hit a stable 4ghz or higher.
I wouldn't worry. If you are sticking with stock cooling just get it to about 3.6ghz and leave it. That would be a nice free upgrade.