Do i need DDR3?

Welcome to Tom's Forum!  

I assume that your MOBO is an AM2+ with DDR2 sooo in order to run DDR3 you're going to need a new MOBO especially if for some reason your current MOBO won't take advantage of the unlocked AMD Phenom II 940 {Black}.

Next the CPU 4-core vs 6-core - my feeling is your money will be better spent somewhere else. Some video software is accelerated by the GPU so beefing up the CPU will not cut the rendering rates nearly as much as using an 'approved' per App GPU.

Q - What video rendering App are you using?
Q - What MOBO are you using?

Wait for bulldozer, which may be released in April, or spend more for the 1155 2500K. I wouldn't change boards just to use ddr3 with your current cpu.

The Phenom II 940 is an AM2+ CPU which means you most likely have a motherboard that only accepts DDR2 memory. You will need memory matching your existing ones for best results. You can't OC that CPU? What motherboard DO you have? Your cheapest option, and maybe the best, is to futher explore overclocking. That and maybe another 4GB of DDR2 memory (if you truely need it) would stretch the life of what you have a little further.

I got a DK Lan Party 790FX M2R.

For some reason when I run the AMD overdrive no matter what setting I pick it never actually over clocks it.

When I go into the BIOs and try to move the multiplier up it always resets to default...

I'm using Sony Vegas 8 Pro for my rendering.

I would like to just get another 4 GB of ram for $60 and OC the process but the OC part is holding me back.

^Simple, uninstall "AMD Overdrive" and OC the BIOS manually. Utilities in general are a VERY poor method of OC'ing, and if you are manually 'tweaking' while the App is 'tweaking' then you'll loose the manual settings.

Vegas Pro 8 relies heavily on GPU rendering acceleration, and there are plug-ins to aide in the acceleration e.g. "GPU decoder".