It will depend on the version PCI-E slot (1.0, 2.0, 3.0) and the mode (x4, x8, x16) not size.
You won't see any performance loss on a PCI-E 3.0 Slot at x8, or a PCI-E 2.0 at x16. Any lower you will start have performance loss with the top end cards.
This benchmark should help.
But for mid-range cards you will see less performance loss. And for low-range, practically no performance loss.
Hope that helps