You can overclock them by increasing the FSB within your motherboards BIOS, but, yes the *multiplier* is locked...

And, if you are able to get the Celeron up to even 3.4G, it will be usually still be outperformed by a 2.4C Northwood core...
 
all intel desktop cpus are locked especially the celerons, only ES samples are unlocked and the older P1's and early P2's are unlocked (or can be unlocked).

Overclocking on the other hand is easy with the right board through fsb overclocking and celerons are usually legends at overclocking, especially the 130nm celerons.