Yup, best to go a DDR2 platform for this budget build. Also for comparison my 8800GTS 512mb maxes out SC2 minus AA with smooth frame rate even in heavy battles, etc. Smooth frame rate being a minimum of 30 fps at 1920x1080, quite acceptable for an RTS imo.
So i'd suggest not going lower then that. Higher if you want to turn AA to 4x or more. Which the 5770 will do nicely. The 5750 'should' also do that, if you want to take the risk, it's cheaper.