There's no way in hell your maxing out the 16x PCI-e 1.0 with that card.
You won't feel much difference at all.
Sure ram will help, but only by small percentages - which you will not notice.
Youll pretty much just waste money.
Id verger your old e6850 + the q35 chipset is choking a bit on teh 6770.
Start looking into a budget i3/SB Build instead and pop in your 6770, you'll see actually use the performance and can live with it til your gpu needs an upgrade if you want more performance.
You can probably get a budget thing going for the same price as the optiplex anyhow.
I'd get a new PSU,MOBO,CASE,i3 and save the rest frmo the old optiplex.
Value is just immense more.