Like Pearlman said, you should Google it first. There's tons of guides out there. But here's what I would do.
1. Download MSI Afterburner to Overclock with.
2. Download FurMark AND Unigine Heaven Benchmark
3. Slowly start to move the CPU Clock OR the Memory Clock. Only do one of those at a time so you know what's making it become unstable or not.
4. Move up in increments of 10MHz or 25MHz. After each time, I would run FurMark for at least 5 minutes. That will show you the highest temp and you can quickly check for artifacts (strange colors/things appearing on the screen).
5. Once you've reached a spot you comfortable with and is stable for the 5 minute FurMark; Run FurMark for 20 Minutes. (If you want, but I recommend).
6. Then, run the Unigine Heaven Benchmark all the way through. Put the settings at DX11 (if you have it), AAx4, Anti-Aliasing x16, tessellation is normal. Then set it to the Highest resolution you have, and keep it in Full screen.
It MUST run through the whole benchmark or else (in my eyes) it's no good.
7. Lastly, Play the highest-graphic game you have for the amount of time you normally do (an 1 or 2 if you can). If it all works and you get no errors or black-screens then your good!
That's what I did, and my overclock is perfectly stable. Hope that helped.