There not much you can do. It doesn't matter if you overclock or not, when you game you increasing the heat output of the components thus the reason why the fans speed increases which in turn increases the noise. You could lower the fan speed of the gTX 770 to a less aggressive fan profile but eventually it will get loud anyway.
After playing games with my case open I saw that the power supply fan was the problem. The power supply fan is louder than my cpu fan, gpu fan, and all my case fans put together. Is there any way to fix this or should I just do an RMA.