You can overclock the chip but it will depend on what cooler you have yes. Overclocking with the stock cooler is not recommended.
You don't need a water cooler for overclocking a good air cooler will get the job done.
Depending on your budget you can this:
Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler
And be ok at about 4.2-4.3GHz with decent temps.
Or you can go for a much better one and get this:
Noctua NH-D14 65.0 CFM CPU Cooler
Will be more than enough for 4.5GHz on the i5.
You should check memory clearance and case comparability before buying new coolers.