I did a few more tests today.
I reset the CMOS of my MOBO by taking out the battery, still didn't work, same symptoms.
I tried putting it into the other PCI (x4 I believe) slot, the computer turned itself off after a few seconds (Same situation as the other PCI slot, except it actually turns itself off)
I tried plugging in both PCI-E PSU Power slots without plugging the GPU into the pci slot, this let the computer fully boot up normally (but then obviously no GPU).
A couple of times I reseated it back into the pci3.0 slot, the computer turned itself off just like the PCIx4 slot after I put it in there, then it went back to how it was before. (Where it turns on but doesn't boot, but without the auto turn off)