Graphics card for a 2 ½ year old pc

Why not give one of ATi's 5 series a go?, and more RAM on XP is pointless and as DDR2 is not getting any cheaper, I wouldn't bother.

I think my choice is between a 4870(1GB or 512MB?) or a 5770.
Is DX 11 support and whatnot worth the extra cost, and I read here in a review that the older 4870 card performs slightly better than the 5770, what would you recommend?

xp 32 bit can only fully utilise 4gb ram including vram on your graphics card so for example say your graphics card has 512 ram so now xp 32 bit can only handle 3.5 gb ram now on new 64 bit operating systems they can use terabytes of ram if you could get that much, but more usable ram will give you better performance in overall productivity and multitasking

the 4870 is a bit better then the 5770 currently but with driver updates the 5770 should surpass the 4870 in time

No, just apply card and do the driver change thing and you should be good to go.

should be fine higher memory just gives better performance on higher resolutions

^ Probably for the best, ATI has yet to post any drivers for the 57XX cards under XP anyway
The CPU is a good overcloacker BTW, and a HD4870 will benefit from boosting the CPU speed, take a look at the overclocking forum for advice.
My experience of the RAM issue is that 4Gb is not entirely wasted but if you do not multitask greatly then it's probably not worth the expense, although it does help speed up exiting from memory hungry games.