PCIe 3.0 provides twice the bandwidth of 2.0, which provides twice the bandwidth of 1.0. There are other differences not usually worth mentioning, like that a 2.0 slot can provide twice the power of a 1.0 slot (only matters if you want to stick a card pulling more than 75W through the slot alone into a 1.0 slot).
Since only the highest end cards can make use of all the bandwidth of a 2.0/2.1 slot, you can put almost any card into a 2.0/2.1 slot and get 100% of its performance. The exceptions would be things like a 7970 or 680/690.
There are also benefits to having 3.0 in doing SLI/Crossfire. There are the same number of lanes, but with double the bandwidth, so you can get a lower end motherboard with PCIe 3.0 and not need to worry about whether or not your card is going to be handicapped by an x8 slot (since it's equivalent to a PCIe 2.1 x16 slot).