OK, there's no reason to think the motherboard will work properly with 3 modules, but it certainly should with 4.
What is it about them that won't work? Do you get error on POST, on the system startup?
When your system boots, it displays the speed of the RAM installed. Does it say 800 or 667? Does it show 2048K when 2 sticks of RAM installed and 4096K when 4 sticks?
One thing to do is to make sure the RAM is receiving enough voltage. In your BIOS, under Advanced Chipset Features, you have a System Voltages option. Chose that and select Memory voltage. Set this to 2.1V when running 2 modules and 2.2V when running 4 modules. Save and reboot.
Assuming your system is showing your RAM as 800MHz and you have set voltage to 2.2V for 4 sticks, this should fix it.
Here's the manual for this computer
http://www.evga.com/support/manuals/files/122-CK-NF66.pdf
And the specs on the RAM
http://www.ocztechnology.com/products/memory/ocz_ddr2_pc2_6400_sli_ready_edition_dual_channel