Assuming you're reading the core temperatures, not just socket temperatures, you're still in the green for thermals. Even when using (hypothetical) 30 C as a baseline, your delta-temperatures aren't too bad; 8 and 28. That means your cooler is doing its job.
http://products.amd.com/en-us/DesktopCPUDetail.aspx?id=772&f1=&f2=&f3=&f4=&f5=&f6=&f7=&f8=&f9=&f10=&f11=&f12=
According to this page, your max core temperature allowed is 70C. That leaves you with a decent amount of headroom. Sadly, for whatever reason, it doesn't list the operating voltages for any of the FX-6XXX series processors. What I would do, is "un-overclock" your CPU, observe your voltages and record them. Whatever the max voltage it goes up to I would consider safe to, incrementally, increase to while overclocking. Be warned, increasing voltage also increases the heat, so I can not stress the word incremental enough.
As kind of a rough example to go off of, I have a FX-4170, which is 4.2 ghz operating at .8125-1.4125V with a TDP of 125. It is a factory overclock of the FX-4100 which runs at 3.6 ghz and has a TDP of 95. (Voltage range unknown to me at this moment.) I realize that this site (Tom's hardware) doesn't like going above 1.3v, but, who knows, your stock configuration might go slightly above 1.4v anyways. :\