Sounds like RMA time, bad luck i know, but it happens.
After all, if you reinstalled windows and drivers and it still does it, somethings wrong.
I have a 5770 and can say that the newest drivers do not seem to have any issues like this(in fact have no seen anything like this with any drivers for that card). I will make sure my 5770 system is on the latest drivers as well(edit...12.6 so latest).
I would check the heatsink/fan to ensure it is clean, but if your temps are 65c load then I do not think you have a temp issue. The 5770 will take 70-80 without any issues.
Did ATItool show lots of artifacts(yellow dots) on its scan as well?