Which PSP model are you talking about?
Sony's newest PSP-like thing, the Vita, has no physical drive, so it can't use old physical game discs. It can use all old games, but only by downloading them through the PlayStation Library. I believe this requires purchasing them again.
The current iteration of what's specifically called the PSP is fully compatible with old PSP games, I believe.
Right. As long as you are not getting a PS Vita or PSP GO, all of the previous games and game disks will be fully compatible. Hope this answers your question (just re-post if you have additional questions).