Airflow. At max settings you are going to stress even a 780, and the result is called heat. This is totally normal, especially after several hours of intense gameplay. With all this excess heat around the gpu, its going to mean that increasingly warmer air is being sucked in by the fans, lowering the efficiency of the heatsink to be able to remove the heat, causing the gpu to get warmer.... And it starts all over again...and again etc till it eventually gets the gpu so hot, it starts to have thermal probes like you experienced. The answer is get more airflow. Increase your cases ability to remove the heat put out by the gpu. This may mean higher cfm fans, more exhaust fans or bigger/better fans. Either way, the heat has to go. It's not uncommon for ppl to even mount a fan to the HDD cage, blowing directly on the gpu if they don't have side panel fan mounts.