The 5770 and GTX260 tend to trade punches depending on the game. The GTX does consume more power as it is a last gen card. Well we ill consider it such because Nvidia doesn't have a new card out yet as ATI does. The 5770 supports DX11 which will be up and coming with some new titles. However, the GTX260 has PhysX going for it that some games really take advantage of. IE, Batman.
Since you are limited by your power supply, the 5770 is probably the best solution. But I would just check out some of the performance review for the 5770 and look at which card performs better on the games you play. The GTX260 is usually included on most of the 5770 reviews I have seen. I don't think it vouches for a PSU replacement though, plus the GTX is usually more expensive.