It depends on the game. If your playing a GPU intensive game then "maybe". If your playing CPU intensive game then "probably"
I OCed my Q6600 from 2.4 to 2.8 and in Flight Sim X I saw a good 10/15 fps increase. Keep in mind that is going to be an abnormal result as FSX is VERY CPU intensive compared to most, if not all other games.
It will help yor framerates, specially the minimum frames, but it won't be anything ground-shattering, except in FSX, Supreme Commander and maybe some other RTSs with lots of units, since CPU handles mostly AI and Physics, even though, since a mild OC will only cost you a couple of pennies a month in the electric bill, I'd say do it!
Yea i reckon, because my current cpu is so crappy (AMD 3500+ 2.2 Ghz single-core), and i have 8800 GT, i once ran the Counter-strike source FPS test with stock 2.2 ghz and oc'ed 2.6 ghz and i actually got a 40 FPS increase!
it depends on many various issues....generally yes, oc is boosting games performance but in your case if you change your old AMD for new E8500 and still paired with 8800GT on 19 inch there won´t be any significant difference running your e8500 at stock (3.16Ghz) or going at 4.0Ghz except some extreme cases (as GTA4 or FSX as mentioned above) but even there you can not expect any miracles. You will be much more GPU limited then CPU limited...your GT8800 and lower res is holding you down.