I've ran memtestx86 through 8 cycles with 0 errors found (15 hours of testing)
I removed all memory but a single stick, pc still froze.
I downloaded GPU testing program called OCCT. I ran a test within 10 minutes 500,000 errors had occurred and shortly thereafter my PC froze entirely.
At this point I feel pretty comfortable that my GPU is pooched and I need to purchase a new one. I know people will suggest switching out the GPU (I don't have another) or maybe running with onboard graphics but I have 2600k CPU with a P67 motherboard so can't.
But is there anything else I should try prior to saying for sure my GPU is the problem? I know it's possible it could be the motherboard but I'm unsure how to test that.
Heat isn't really an issue, I should have noted that I've tested the fan and monitored the temps of the GPU. The highest temp I've seen is 84 degrees while the fan was running at 90%. Not super cool by any means but well within normal operating temperatures.
It's an ATI 5870 so it's no spring chicken, had it since 2011.