The Radeon 5870 would definitely be more capable than the 5770, though both are getting a little old now. Do you currently have one or the other or are you looking to buy a new one? Also, what PSU are you running and what CPU do you have?
This is going to be my first custom pc build. I am planning to use a Intel Sandybridge i5-2500K Unlocked Core i5 Quad-Core Processor (3.30GHz, 6MB Cache, Socket 1155), and a XFX Pro550W psu. I just need to decide on a good graphics card for running the game on high/ultra.