Per core, the phenom II will generally perform the same as a core 2 that is 200mhz less.
ex, a phenom IIx2 3.0GHz will perform like a core 2 duo at 2.8Ghz.
Gaming depends more on your graphics card, but this varies from game to game, so you would need to give us details on the rest of your system. Games are becoming more core optimized though so I would get the x4 totally.
I would say it's pretty much worth it, especially because even if you don't like the upgrade (and you will) you have nothing to lose, since the Phenom II X4 CPUs are dirt cheap at the moment, and plus, you can upgrade to a new X6 CPU because it will be compatible with the current socket. There is no way PII will be slower in anything than C2D, that's for sure. However, you will not notice an upgrade in all things you do on your PC. For example, audio/video encoding and rendering, some games and many other software that is quad optimized will now run like a dream compared to your C2D, but casual games and normal PC use along with simple software will feel the same like before.
To sum it up, I think you can get a pretty decent deal on something like Phenom II 955. It will be a huge upgrade in some sectors, smaller in others. The real thing is that the platform you are getting is very versatile and it can be upgraded later.
So, you now want a good board. I suggest anything from Asus, like M4A79T Deluxe (the old classic, but still one of the best AM3 boards), or Crosshair III Formula as the best possible option. The CPUs you want to be looking at depending on your budget are Phenom II 965/955/945, or eventually the weaker 925 (it can hardly be found except in pre-builts).
Cheers and hope that helps.
PS: And of course, to warn you not to listen to the BS talk about PII being slower than C2D in dual optimized apps "clock for clock" (boy, how I like that phrase) as it simply isn't true. If anything you will see a tiny boost in dual optimized apps since the architecture of PII is superior, but it will really shine in multicore optimized applications.
Hey, really sorry for not replying sooner as I am not so active on these forums.
The motherboard you mentioned has some nice specs, and I think it might actually be an excellent mid-high range option. The chipset is excellent, as well as support for high speed DDR3 memory. It has two PCIE slots as well, so I'd say it pretty much has everything it needs. The only thing that's bothering me is the southbridge, but since you are getting a quad CPU (you won't need to unlock cores if you want) that board gets my recommendation.
Perhaps you should also consider the suggestion above, but in my opinion it is just postponing of the inevitable. You should get yourself a new platform that supports new technologies right away for a bit more money and you will be more satisfied in the future.
You will likely not notice a difference between the E8400 and the Phenom II 945 with a 5770. Your system is still plenty good for gaming. When you oc your CPU and you gain no frames that means you are GPU bottlenecked and your CPU is up to par (and/or overkill).
DDR3 isn't going to make a difference in gaming (less than 5%) and the core2quad CPUs (q9400 and above) match Phenom II in gaming.
I'd buy the 5770 (keep your current setup), oc the CPU until your frames stop going up (which won't be much, the 5770 doesn't require much CPU power). If you want to add a 2nd 5770 in the future then I'd grab a Q9400 or better and OC it to ~3.4ghz, or upgrade your whole system to Phenom II.
On dual core optimized games (which is nearly all games) the E8400 will likely out-perform the Phenom II, assuming you don't have a bunch of stuff running in the background. Newer games are becoming quad core optimized (sometimes) though so its something to look into.