I've been using only MSI Afterburner, but I completely uninstalled it and the issue still happened. I pulled up nvidia inspector, and it looks like its in Performance level 2 (P1) state, which the normal before the issue happens is Performance level 3 (P0). Its so stuck that the GPU doesn't even do the low power 133mhz self underclock when on the desktop like it normally does, its permanently stuck at that 540mhz until restarting. I can use nvidia inspector to change the performance level 2 clocks, but then it just crashes as soon as I launch a game.
In case anyone else has similar system specs that might me causing this, I'm using a Clevo P670RE3 laptop, with 970m, g-sync, 75hz display IPS panel. The issue appears to happen faster on older games that are less demanding. It takes quite a while to start causing problems when I play modern games that stress the GPU. Alt tabbing mid game, or force quitting games also seems to cause it to happen suddenly.