You technically can overclock the i7-2600, but with a locked multiplier, you have to use other methods that are far riskier than it is worth. Even if you went for it, the OC would be minimal.
That said, let's say you had a 2600k. I wouldn't recommend overclocking it at all on the stock cooler. In fact, I'd recommend against using the stock cooler on the non-K cooler, without overclocking. The stock cooler is kinda sucky.