this is a tuff one , if there isn't any scrape or rub issue when you press with even pressure by hand , you wouldn't think there should be any issue , while playing .but from how it sounds as of now , i would say the voice coil may be damaged or more likely the alignment is off . meaning the voice coil does rub when speaker plays .you could try to refoam the bad woofer and see if you can get it right or maybe you want to look for a replacement . if you didn't remove the dust cap and shim the voice coil there is a good chance alignment could be an issue . 1 mm or less can be all it takes to upset this alignment . now if you are positive that you got the alignment right then the voice coil possibly could have been mechanically damaged by being played without the surround in place as the surround helps maintain the alignment while the woofer is playing , if this is the case look for a replacement .