From the very beginning of Intel/AMD processors the speed rating was based on how the batch tested. For long-term stability, processors are usually labeled at a speed 1-2 notches lower than their highest stable tested speed. These processors have never had a clock chip embedded in them; the clock has always been determined by the motherboard. Starting with the 486DX2 series, chips speeds were determined by the motherboard Front-Side Bus (FSB) and the internal multiplier (2x in this case). Current processors have much higher multpliers (8-20 depending on the proc). Due to the ease of overclocking, some unscrupulous vendors started buying the less expensive processor, overclocking it via the multiplier, labeling it as the higher speed processor and selling for a higher profit. Starting with the Celeron 300A, Intel locked the multiplier of all of its chips. AMD followed suit with the Athlon, but left options open for enthusiasts. Now most overclocking is done by simply increasing the FSB, but some processors (currently Athlons & Durons only) can have their multiplier unlocked to be overclocked even further. Other tricks are often used to get extra speed from inexpensive processors; vCore overclocks and water or refrigerated cooling have also become more popular.
Mat is right on the subject of back filling the market; Even when all yields are testing as the fastest chip on the market, not everyone is willing to pay for the fastest chip. So the manus will lock and label them at the market demanded speeds. As faster CPUs ship, these lower speed procs are discontinued. If the manus release the faster chips too soon it significantly degrades their profits on current shipping chips and hurts them in the future if they can't release a faster chip later.
I thought a thought, but the thought I thought wasn't the thought I thought I had thought.