Stock setting is 3.3GHz with a turbo boost of 3.7GHz.
I'd say a healthy OC is anything 4.0GHz and up. Thats an increase of 700MHz, or 21% over stock speeds. That said, most chips out there can pretty easily hit 4.4 - 4.6GHz at safe voltages with a decent cooler.
My preference is for faux liquid coolers like the Corsair H50, H60, H80, and H100. They're an easy install, take up less room in your case, and exhaust all the heat straight outside which lowers overall case temperatures. I've been told they're more prone to failure than air cooler, but the reviews are generally good and I've never personally had an issue.
Big air coolers from Noctua and the Hyper 212 from Coolermaster are also very popular. Basically, what it comes down to with air coolers is that more heat sink mass, more pipes, and more fans equals a cooler running CPU. From there it comes down to budget and aesthetics.
Avoid using the built in overclock feature on boards. They often (99.9% of the time) jack up the voltages to the maximum safe levels which is usually completely unecessary and can shorten compenent life. Overclocking should be performed manually via turbo boost or multiplier settings utilizing user choosen settinging when possible. Best advice is to do a ton of reading, then when you get sick read some more. It's very important to know what your're doing when going for high overclocks and incorrent voltage and BIOS settings can have system destroying consequences.