Make sure your RAM is correctly configured in the BIOS. MANUALLY set voltage to mfg spec, do NOT rely on mobo default settings. Also check the timings and make sure they are at spec.
I second notherdude. I have 4 gig of OCZ DDR2 800 and my ASUS board would not even POST unless I manually set voltage for RAM to 2.1v from the stock 1.8v default on the board. OCZ spec is 1.9v-2.1v, check with your manufacturer for their specs. I had to put in old RAM, change BIOS, reload new and now it runs like a champ PS Also make sure your power supply is big enough for all loads you have running.
my ram is perfectly sable (i even have it underclocked) it is defiantly a software problem
OK . . .but if your RAM is underVOLTED it might not be stable at any speed. There is no reason Vista should choke on 3 gig of ram which is why we focused in on hardware. By all means make sure that it is getting mfg spec. We see a lot of memory that runs at 2.0- 2.2 volts these days but MOBOs default to 1.8 volts and this often causes this kind of problem.
It is well known that many MOBOs have a hard time running four sticks. Ways to fix this are to run at slower bus speed (just the RAM, not the CPU), relax the timings (if 4,4,4,12 try 5,5,5,15) and bump the voltage. Alsio sometimes a bump in the Northbridge violtage.(not necessarily all of those things but one or some combination
The fact that you pass memtest and Prime 95 is good but that does not necessarily mean you are stable in windows. Maybe, maybe not. But I would explore the RAM settings because I cannot think of any better thing to try. The clue here is that the system is stable with 3 sticks but not four, this argues against a software problem. Anything is possible of course.
Also at least try the hotfix I linked. If it does not apply to your system it will refuse to install