There are so many things that can be wrong, is your power supply good enough to support it while in full load? Do you have any RAM errors? Do you have an overheating problem?
Tested with memtest for around 4-5 hours, Not a memory problem. 600W SLI certified power supply, The load for the 5750 is only around 86W, The CPU only gets up to around 52 degrees Celsius, the video card around 56.
I've tested with an old nVidia card (6600GT, old i know) but it ran without a hitch with the nVidia control panel.