Do not overclock without a 3rd party CPU cooler (i.e. do not overclock with the stock cooler).
For ASRock boards, you should just be able to go in and leave everything to auto except the multiplier. You can get a good overclock (but probably not max stable) that way. I got 4.4GHz stable during testing when I first got set up.
Start at probably 4.0GHz (safe side) and run Prime95 (large FFT) or OCCT to put a load on the cpu. Watch temperatures. If stable and cool for your tastes (I'm not interested at higher than 70C), up it by 100Mhz (e.g. change the multiplier from 40 to 41) and repeat. Do this til you get a blue screen or too much heat, then back it down 100Mhz and you're done.
100% agree with J_E_D_70, an aftermarket cooler is a MUST lol. Luckily you can get coolers that will bring you to a decent overclock for pretty cheap, but as far as overclocking help goes, every processor is different, so nobody can give you an automatically stable set of numbers, my stable number will be very different from someone else's. This is the guide I use and reference often, and the people on that specific board are very good at answering specific questions you may have after going through the guide step by step for yourself. Hope this helps