The ambient temp of the room and the case make a huge difference.
The cooling of the CPU is the cooling of the water, the cooling of the water is the amount of heat you can take away by blowing air through the radiator.
Hence there will be a point of balance between the air outside the case, the air inside the case, and the drop that can be experienced in water temp by the radiator. 8O
The real deal is to remove the heat from the case. Usually this means the radiator has to be on the outside, or the fan blows through the radiator to the outside.( not as good as radiator on the outside but the one you have?.) If most of the air through the radiator vents out of the system it should be ok.
I would recheck the setup. Note you have not mentioned the CPU.
Here IS A LINK. JUST LOOKING AT THE CONCLUSION the 100 watt for the "big water is 44, and 150 watt is 53".(celcius)
http://www.xbitlabs.com/articles/coolers/display/16watercooling.ht
Did you use Artic silver 5 --this will return you 2 deg cel approx.(worth it)
The ventilation of your system is weird
120mm in and 80mm out?
also 120mm out through the radiator then through a hole ? (if thats it then that is fine.
BUT you do not have enough air coming in! by 1x 80mm fan.
You are creating a vacuum (it seems) which will stop correct cooling of the radiator., or you simply do not have enough air going through.
Let me say , 120mm 9 bladed fans, 220cu feet / min. at 65 dba.
Delta 120mm Ultimate High Speed Fan TFB1212GHE
Keep them at low RPM for modest air flow but exceeding that of other fans.
I would artic silver your cpu, get another air input into the case, and if the 120 is not venting to the outside world through the radiator do that, failing that I would mount the radiator outside.(a bit drastic but that is what I am doing on a small antec lanboy.)
I also believe you have the wrong water set up, see the review.
i'm shoping for a setup, 1 highflow swiftech pump, 1,x120mm radiator,
1x high speed 120mm fan for radiator, 1/2 inch barbs, tubing, 1x high flow
reservoir, and I have yet to source the water block. The first reservoir and the second will be mounted at the back, exit fans blowing through.
Hope this helps