I can get my memory to DDR2-870 4-4-3-9 stable. But as soon as I try to go for 900, or even 880 I get errors immediatly. No matter what I do. I have tried upping my NB voltage, RAM voltage basicly everything. Even 2.4v VRAM does not cut it. 2.4v cant even make DDR2-880 stable, even with 4-4-4-12 timings.
My CPU is not the bottleneck, I have made that sure.
OCZ Reaper HPC PC8500 2x2GB
Asus Rampage Formula
Q6600 G0 (435x8)
That RAM was factory-overclocked when you bought it. I'm not at all surprised it won't go any higher.
To avoid this problem, and give yourself some headroom, start with RAM that operates at its advertised timings on the JEDEC standard 1.8V.
Well, the timings weren't as tight as yours, but I "accidentally" oc'ed my Mushkin DDR2-800 to DDR2-960 and my system didn't so much as hiccup over it. I say "accidentally," because I'd forgotten to reset a ratio to keep the RAM at 1:1 when I oc'ed my CPU to 3.2 GHz. I'm pretty sure my RAM is still running on the stock 1.8V.
Higher voltage doesn't always mean faster speeds or better timings for ram. Each stick is different. I had Patriot Viper ram that was rated at ddr2-1150, 5-5-5-15 @2.3V. But it would Always fail memtest86+. After much troubleshooting, I found that the ram passed memtest86+ 1150mhz, 5-5-5-15 @2.0V. So, for some ram too high of voltage will cause worse performance.