If anything, DDR2 as the more mature platform would be more reliable. Running your RAM at a faster speed then your FSB doesn't really help your computer perform better. And because DDR2 is already faster then your FSB, there really isn't much point in DDR3. Nehalem and tri memory controllers will probably change this, but at this time, DDR2@800MHz really is all you need.
DDR3 is basically what DDR2 was. Double the latency, double the clock speed, lower the voltage. I understand DDR3 actually adds some new things into the mix, I haven't memorized them yet. If you can run both, I'd stay with DDR2 right now. Wait for DDR3 to continue to fall, and buy it if it ever makes sense.