That might help some, but it won't make a huge impact. is your e8400 overclocked? If not, invest in a good aftermarket cooler and thermal paste and push that cpu as far as you can. I have a core 2 duo E6850, the previous gen of your processor, and have it running at 3.6 Ghz. That will cost you about $60, and give you the most tangible performance gain. Otherwise, there isn't too much you can do without upgrading your core components, your GTX 260 is already outpacing the CPU. With the i5's out now though, save up another $200 and you have the option to get a basic i5 / p55 setup that will run circles around core 2 duos.
no, you'll have to get a new motherboard, but the p55 motherboards are way cheaper than the x38 boards. $150 should get you a solid board, and $210 is the current price for an i5. You will need at least 4GB of DDR3 RAM, but $90 should cover you there and reaches the estimate of $450. OCing is still the better choice if you haven't done that for your e8400 yet.
I'd OC the CPU w/maybe a Xigmatek s1283 aftermarket HSF and possibly upgrade the GPU to a 4890. The GPU upgrade will make some differences, especially if your running at higher resolutions. The GTX 260 is a good card, so you don't really need to upgrade it, but since the 4890 is coming down to about $150 or so, you could afford the GPU upgrade and also get the HSF for under the $250 limit. What resolution are you running at? If your below 1680x1050 than the GPU upgrade won't make sense.
Guess i'l get the cooler and OC my e8400. But the motherboard isnt a very good overclocker from what i saw in other forums. And can my rams survive the overclocking? suppose i manage to overclock it to 3.8 or 4GHz and its stable, wil i be able to use it everyday?
how about getting a xfx gtx 260 black edition for sli? saw one for $189.99 in newegg. Will it be worth buying as i run at 1280*1024
check the overclocking guide sticky in the overclocking forums, they'll give you the basics on how to OC a core 2 duo. They aren't written specifically for your board, but if you have most of the BIOS settings that are similar in name to the ones in the guide, you should still be able to get it stable. You might not be able to hit 4Ghz, but 3.4-3.6 should be relatively easy. Also, if you can manage that, you can try to push it further if your temps and voltages allow.
Getting more graphics cards won't increase speeds at this point for you, you need more CPU cycles to fully utilize your GTX 260 even. If 1280x1024 is all the resolution you need, you should still get acceptable frame rates for the time being.
Zalman has some good coolers for your board type, that's what I'm using for my C2Duo. Noctua and Thermalright make really nice coolers, but they tend to be the most expensive. OCZ is on the cheaper side and I've had good luck with them in the past, but I didn't do any aggressive overclocking on that system. Pretty much, any aftermarket cooler will do a better job at cooling your CPU than the stock one, how much cooler they keep things, and how loud they are will determine the price.
zalman looks good, might get one, only concerned about the size now.
one OC related question about my board: how do i change the fsb : dram ratio? there is no option for it. But i saw an option to change the ram bus itself. should i change that? Or should i change the ram timings to sth higher?
the zalman with 92mm HSF should fit in your case, that's pretty much the same one I have, and my case also happened to be a thermaltake armor jr., should have similar dimensions to your m5. I just moved the system into a HAF 922 case because I was having crowding issues with drives and GFX cards.
well I don't have access to my desktop right now, but using the zalman cooler, I'm able to overclock an E6850 (stock 3.0Ghz) to 3.6Ghz. Idle temps are usually around 37C, not completely sure on load temps, but I think around 63C is the highest I've seen on it.
At a 1280 x 1024 resolution the GTX 260 is just fine. OC'ing your CPU with DDR2 800 mHz will be just fine. You would be going from a 333 mHz FSB (9 * 333 FSB = 3 gHz) to 400 mHz FSB (9 * 400 FSB = 3.6 gHz) to get an OC to 3.6 gHz on the CPU. The FSB will go from the 333 mHz (DRAM is 2 x FSB, so you'd have 2 * 333 mHz = 667 mHz DRAM speed) to 400 mHz (2 x FSB = 2 * 400 mHz = 800 mHz DRAM speed). This means no OC'ing needed on a 800 mHz RAM. This is why I suggest using 800 mHz DDR2 at CAS 4 speeds. This will be the most that your going to need.