Yes, there are differences between the first and second generation of any processor. It would mainly be in power usage and performance. However, it may, and has, extend to architectural changes with the development of new manufacturing techniques.
The bigger, and more pertinant, question is whether that processor is compatible. In recent times Intel has produced the LGA1156 Platform with the first Gen i3/i5/i7 mainstream processors, the second and third gen processors of the same range came on the LGA1155 platform which was incompatible with the first.
AMD, on the other hand, has used the same physical socket (AM2/2+/3/3+) for a number years now. This has produced a more stable platform which supports a larger variety of processors. It also has the problem that you actually need to know which revision of the socket you have and the power capacity of the motherboard to know which processors it can actually support.
It would help if you're looking at any particualar processor or just asking in general.