I've had this card for nearly a year now with very few problems (the only problems I have had were very infrequent screen flickers, likely due to my overclock). What I got tonight was something, like I said, I have never seen before, period, and I honestly cannot even begin to imagine what caused it. I rechecked all my settings, as you suggested, and everything appears to be normal.
You are right, this card is definitely beginning to worry me. I might RMA it anyways just for that screen flicker problem and see if they accept that..