The 2500k has an unlocked multiplier making it easy to overclock with a motherboard that has either the P67 or Z68 chipset. It also has the faster Intel HD 3000 graphics core which is basically equivalent to the Radeon HD 5450.

The 2500's multiplier is locked and overclocking potential is very limited; 300MHz to 400MHz at most. It has the slower Intel HD 2000 graphics core, faster than the older GMA 4500M though. Good enough to watch movies. There's also some circuity that allows a corporate IT dept to more securely connect to a PC with an i5-2500. Lastly, there are circuity that prevents hackers from copying certain information in the RAM and on the hard drive. Great for the corporate environment with sensitive data to protect. Not much use for the average home user.
 


Jaguar pretty much gave every detail. The biggest thing is the 2500k can be over clocked pretty high. The regular I5 can be over clocked but not as much as the 2500k.
 
the core i5 2500 and 2500k share almost everything except the unlocked multiplier.
the core 2500 supports vt-d, the 2500k doesn't. the 2500 also supports txt and tpm on b6x and q6x motherboards (for corporate use).
the core i5 2500k can be overclocked by changing the multiplier when used with a pch that allows cpu overclocking i.e. p67 or z68. it should also facilitate ram overclocking using intel xmp.
the core i5 2500 can be overclocked only a few mhz using the turbo multipliers (a z68 motherboard) for single core max turbo. it's multiplier is locked.
the 2500k has hd 3000 igpu with 12 execution units. the 2500 has hd 2000 with 6 eus. both are sufficient for everyday basic tasks like text editing, web browsing, movies etc.
for gaming, a core i5 2500k would be preferable for it's overclockability. people have been able to oc the 2500k from 4.4 to 4.8~5 ghz (it's got 3.7 ghz max single core turbo). some cpu-bound games benefit from the extra clockspeed.
if you get a core i5 2500k, get a good z68 motherboard with decent (dual x8) cfx/sli support. also get a good cpu cooler like coolermaster hyper 212 plus/evo or xigmatek gaia . the cooler will help dissipate extra heat from the cpu when it's overclocked.