You need an aftermarket CPU cooler to overclock.
A good time to overclock is when your CPU is at 100% usage during gaming. This means it is bottlenecking the GPU. If it is not at or close to 100% usage in games, overclocking will not really do anything.
Check your CPU speed using a program like Coretemp which will also show your CPU temperatures.