Get the best video card you feel comfortable buying.
The video card is the most important component for gaming, particularly at higher resolutions.
Yes, your dual core cpu is a bit dated by today's standards, but any current video card will be a big improvement, and can be carried forward to a future build. I would expect that your cpu is strong enough to give you improved performance.
Try these tests:
1) Run your games, but lower your resolution and eye candy.
If your FPS increases, it indicates that your cpu is strong enough to drive a better graphics card.
2) Limit your cpu, either by reducing the OC, or, in windows power management, limit the maximum cpu% to something like 70%.
If your FPS drops significantly, it is an indicator that your cpu is the limiting factor, and a cpu upgrade is in order.
It is possible that both tests are positive, indicating that you have a well balanced system, and both cpu and gpu need to be upgraded to get better gaming FPS.
There is some value in 8gb of ram, but unless you do lots of multitasking, the money can be better spent elsewhere.
I know that my components are quite dated but if I do upgrade to a higher end cpu will my mother board limit it's performance? I am really hoping I don't have to scrap everything but I would like to play BF3 on descent settings. Thx
You know what I would sell your build. Honestly if your gonna put 500$ into which sounds like you are anyways why not get a new i5 2500k next month and build a new system for about 900$. If you sell your current rig for $350 that would help you out. The cheapest 775 quad core goes for around 150$ so that will put you with in striking range of sandy bridge. Don't invest in old technology, future proof yourself for a couple years. I would not suggest spending any money on your current build!
Since you already have a decent board it's not a bad idea to get a 45nm Quad Core and overclock it. Considering that Q8000 CPUs aren't that great and Q9000 CPUs are often overpriced you may have to go to with eBay on that one. If you don't like eBay for whatever reason then I would say go with a Q9505S and overclock it. If you don't want to invest that much money in a CPU then you have to at least consider overclocking your current CPU to avoid weighing down a modern GPU.
There isn't much point in getting 8GB of RAM for gaming unless you really have alot of stuff in the background eating up alot of memory. For a GPU go with a single GTX 460 or Radeon 6850 would be as high as you should go with an overclocked 9505S. If you only overclock your current CPU then just get a cheap 5770 as they are as low as $120 but are still twice as fast as your current 2900. Even if you only go with a 5770 I would replace your Rocketfish PSU since I wouldn't trust any Rocketfish PSU with shiny new components.