There is no need to excuse towards us. It is your CPU you are endangering after all. We can only tell you that it matters not whether you took it out for one second or one hour. What counts is that you broke the paste connection between CPU and cooler, and once that is done, the paste needs to be replaced, because there will be air bubbles when you simply put the CPU back in place, and the cooling effect will no longer be what it used to.
Now make your decision as you like, but do not ask us for absolution, because there is none. The paste needs to be replaced, and you can decide not to do it at your own risk.
I would like to remark that I find your behavior pretty childish. First you panically create a thread asking whether you damaged your precious CPU by pulling off the cooler, and when we assure you that everything will be fine if only you go to the nearest computer store and purchase a new thermal grease for like $6, then that is too expensive, and you rather want to risk your machine. How silly is that?!?