Dang. RAM voltage would've been my first, second and third guess, followed by the PSU. What ARE the specs of the RAM, and what settings have you tried?
If you haven't updated the BIOS version and motherboard drivers from the ones that came on the CD, you might try updating them to the latest and see if that solves anything. Who knows; there could be some quirk in the original versions that causes problems with your particular setup.
You might also look at other software issues, although that could be just about anything. A couple of the more common ones are an antivirus program causing resource conflicts when it tries to start up some background task ... or sometimes drivers for peripherals causing conflicts of their own. Hard to say if/what it could be if it is a software issue, but those would be a good place to start.
I agree that the RAM voltage fixes freezing problems 80% of the time. I would also like to know exactly which "DDR3 2x2GB" RAM kit you have. How long did you let Memtest86+ run? It has to run multiple passes to fully test the RAM. The best way to do it is to let it run overnight.
What hard drive do you have? Did you do a fresh install of the OS on this system?
Crud. That RAM should be about as uncomplicated as it gets, so I doubt it's the problem. It sounds like you've checked the obvious issues. Haven't ruled out the CPU, motherboard, video card or hard drive ... so if there's a way you can try any of those in a different system, or vice versa, that seems like the next logical step.
Time for a cleanup i guess! Take all your parts out of the case and disassemble them. Then only use the necessary components and built your mobo on a carton (or something to avoid shorts and/or static) or something and see if it is working there, if it is then keep adding other stuff one at a time and test until you find the culprit. You may find a faulty memory module, or RAM bank, or even an incompatible memory combination (module 1 in slot 1 does not work, the other way they work fine??). If you still can't come across the fault then it could simply be that there may be a short in your case (something touching somewhere where it should not) and now that you took it out of your case it is working fine.