Can you give me the link or full description/name of the liquid cooler? no way of knowing otherswise...
However, I can give a good bet that any liquid cooler is already better than average air cooling so yes, you should be able to safely get to 4Ghz. Best way to find out is to test. Do the overclock and stress test it while checking temps... if they're good, you're set! If not, you know you need to upgrade the cooling. Most likely the temps will be fine at 4Ghz, its past that when they'll start climbing high.
Will the power supply suffice? It should be fine. Wattage-wise you're set to go with the setup and performance you're looking for. Quality? can't say, you didn't name a specific psu model and brand. If its at least 80 plus bronze certified and made by a reputable brand, the quality should also be fine. So all in all, as long as the power supply is decent itself, 820 watts will be more than enough.
Hope this helps