The AMD yes, the Celery, NO. You will need to raise the FSB and voltage to overclock it. The AMD black editions you just simply raise the multiplier until you are happy or it won't boot.
Also I must point out, the AMD you have chosen is a little outdaded and does not include a heatsink / fan. By the time you buy the heatsink you might as well get a better CPU that includes one. Take a look at this CPU... it's better, unlocked and has 6MB L3 cache. http://www.newegg.com/Product/Product.aspx?Item=N82E168...
Now as for the motherboards... if you want a good setup, I don't know your budget yet, but try to get something with the newer chipsets like the 880 or 890. For the Intel chips, you will need to buy a board that is setup as overclocking as the main feature. Many Intel boards are a little tricky, especially if you are trying to OC a Celery. The multiplier is locked, so the board will need a good chipset and BIOS to support advanced overclocking through FSB and voltages.
By extreme I mean liquid nitrogen. Don't worry, I got all that worked out. Could you please recomend a good motherboard for both cpu's? Also, if the celeron isn't good, can someone show me a good intel processor with similar specs that I can over clock with? Thank you.