Is it a square or rectangle with an opening in the center? if so, then it is a gasket that surrounds the chipset. Reuse it when installing the new heatsink.
Same thing happened to me a couple of years ago. I kind of stuck it on a new forged copper heatsink and hoped it would stay on long enough for me to apply thermal compound and secure the new heatsink. It did. Without the gasket your results will not be as good.
If it one solid piece with no opening in the center? If so, then it is a thermal compound pad. Replace the pad with the thermal grease that came with the Zalman heatsink.