You're all fine. Don't worry. As long as it's working properly, and you're not having any sort of issues you're all set. Just remember there's a little tab that you press to allow the graphics card to release without quite so much force generally. Also, always remember to pull it straight out and not at a weird angle or anything like that.