What makes processors better?

Ok.. I was asked this question by somebody and though I skimmed over what I felt was the correct answer. I thought about it and I don't think I understand well enough.

Say we take an Athlon 1500 and compare it to an Athlon XP 1800+. What is it that makes the 1800 a faster processor. I understand about core changes... AMD develops a new core that is optimized, or has more cache, or whatever... but what typically happens to make one model of a processor able to run at much higher speeds than another (with the same core) -- without melting a hole in your case.

In theory, you could take an Athlon 2000 and an ample supply of liquid nitrogen, some tweaked voltages, and get it to run faster than an athlon that is meant to be clocked at much higher speeds.

I know AMD has gone through several cores with it's Athlon XP line, the 2200 had a revision, they upped the FSB speeds for 2500's and up... but what else do they change? Do they just get better at the manufacturing process and are able to yield higher counts of processors that are able to be clocked at higher speeds? Are there tweaks made that allow newer processors to be clocked at higher speeds? Is it a little of everything? What kinds of tweaks to chip manufacturers make? I know processors run hotter these days, but the clock speeds seem to outgain the thermal increases - right?

Thanks in advance