first try to check the exact model of it lets say you have the gtx 650 ti then try to google other overclockers record on how much are they able to tweak their cards the try to increase your core clock and memory clock a bit lower from them then if it works youre good to go for the next phase. try to benchmark it if itll cause any error if it does try to lower it a bit more, then benchmark again, (take note
make sure that you have a adequate psu and the gpu that you have is not an entry level gpu. - david gilinsky