For that kind of setup, I'd go with a Corsair 450w unit, will provide more than enough power, is high quality, and fairly inexpensive.
As for the RAM, games rarely use over 1GB, but some are known to use at least 1GB. I always recommend getting as much RAM as you can afford. It may not lead to a big performance boost, but it may smooth out your minimum framerates, and your OS should be a bit snappier.
The Q6600 is still a good processor, even with the i3/5/7 now taking the crowns. If you can, I would run your FSB up to 300-333Mhz, which would overclock you to 3.0GHz-3.33GHz, and would be a nice performance improvement. Just be sure to change your RAM divider to get it to stay at or around 800MHz (PC2-6400).
If you are using a 32-bit version of Windows, 2GB is fine. If you are using a 64-bit version, going up to 4GB will improve performance overall. Corsair makes a decent PSU, as does Antec. As jedimasterben says, get at least a 450w model. Go to Newegg and look for the best offer (and free shipping if possible) and customer ratings.
The Q6600 will not bottleneck given either the 4670 or the 5770 and should perform nicely with either card, OCed or not.
well im using 32 bit, but my buddy has a window7 copy 64 bit, so ill be adding 2 more stick, also what do you mean by "Just be sure to change your RAM divider to get it to stay at or around 800MHz (PC2-6400)."
Replacing a fan is a no brainer. Remove the old one, measure, and order a new of the same size. Seems you have some fan issues with your config. Based on that, I would be careful with the OC, you may cook more than fans.
Also, keep in mind that you can't upgrade from a 32-bit version of Windows to a 64-bit version. That move will require a clean install and a valid license/product key.
Look in the overclocking section of this forum, there is a comprehensive guide on overclocking a socket 775 quad.
When overclocking its your load temperatures that are the critical ones, not the idle. You need to download prime95 to stress your CPU 100% to find out your load temps, and if your cooling is good enough to give you headroom to overclock into. Read the guide anyway, it will explain it.
If the fans in your Ultra PSU are dead, it's only a matter of time before it overheats and fries, possibly taking other parts with it. Add my voice to those suggesting you replace your PSU. A quality 400W-500W model would be sufficient. Look for one with full range active PFC (no little voltage switch) and 80+ certification. Antec, Corsair, Seasonic, and Enermax are among the better brands.
I am actually using an Intel Core 2 Quad Q6600 @ stock 2.4 ghz, Gigabyte HD 5770 and 4 gb of DDR2 ram (which is actually 3.3 gb of ram since I am running 32 bit Win 7)
Been playing Bad company 2 on High setting and my 22" monitors native resolution, 1680 X 1050 and its working great.
You shouldn't have a problem just make sure you have sufficient cooling, i.e. intake and exhaust fans, good cable management and a big enough case/chassis.
And I am happy with my Seasonic M12II 620W PSU. Its two +12V rails each have 24 amperes which is enough for 2 Gigabyte HD 5770s I think (according to the manual a single Gigabyte HD 5770 needs at least 24 amperes on the +12v rail ).