yes replace the power supply 300 watts is not enough the best you could get is the 5670 and that pushing it....and yes the 5770 well be a huge upgrade from the 4670/4650...but you need to upgrade your power supply (even a antec 400 or 380 watt well be more then enough)
I believe Dell mid size and larger desktops use standard ATX power supplies since around 2007 therefore you have lots of options open to you. And yes, you need to upgrade the PSU if you want to install a HD 5770.
To the best of my knowledge, Dell still uses Fortron Source (FSP) as their OEM for PSUs, so here's a FSP 500w PSU for $45+ $9 shipping:
In that price range I really would not recommend any other manufacturer. At higher price points I would recommend PSUs made by Seasonic since they are basically the premier PSU manufacturer. Or specific PSUs that Seasonic manufactures for other brand names like Antec, Corsair, PC Power & Cooling and XFX.
A good 450w PSUs should be enough for your system and the HD 5770 with a bit of leeway for other future upgrades.