rolandzhang3 :
the 7750 would make a good replacement, or the 7770 would give you a nice little added performance
I'm afraid the perf increase isnt that great, I'm currently thinking of upgrading my own 5770 and thinking that anything below 7850/650tiBoost level of performance would be waste of money... 5770 is quite even with a 7750 in many of the reviews/comparisons I've found, winning some losing some:
http://www.anandtech.com/bench/product/538?vs=535
to op
the fan speed sounds really low, I've got one from ASUS with similar cooler and the fan on it ramps up to around 3500rpm under heavy load, so try a custom fan profile as Roland suggested, also make sure the fan isn't jammed or anything so that it spins freely when you manually spin it (it should wobble a little before stopping completely as well)