Performance per clock is improved with each generation by 5-15%. That's the most important part to most people.
Different generations also use different sockets. Second and third (Sandy Bridge and Ivy Bridge) use socket 1155, while Haswell and upcoming Broadwell use socket 1150. The E-series use socket 2011 or 2011-v3.
Starting with Ivy Bridge, Intel started uses thermal paste instead of solder to connect the die to the heatspreader (the metal part on top). This has resulted in significantly higher and more uneven temps. Haswell-R (Devil's Canyon, i5-4690K and i7-4790K) use much better thermal paste, although still not solder. Haswell-R has improved temps by about 10C compared to regular Haswell.
Haswell (and Broadwell) have the voltage regulator on the CPU itself, instead of on the motherboard. This has again raised temperatures compared to Ivy Bridge and Intel will put the voltage regulator back on the motherboard where it belongs starting with Skylake (the gen after Broadwell).
Newer generations also have support for more instruction sets. For gamers this doesn't really matter.
At the end of the day, if you're building a new system you should just go with Haswell or Haswell-R. They are not a lot faster than earlier gens, but faster none the less. If you're upgrading however I wouldn't bother if you have a Sandy Bridge i5 or better. The gains are just too small to justify it, for gaming anyway.