I am not sure why this happened, but I think it may be a problem with spreading the as5, applying too much, or maybe you are using it without cycling the ps3. If I were you, I would let it cure with the power on and play it for a few hours and turn it off for a couple hours and repeat the process. The heat speeds up the process, so play a hardcore game for a few hours to get it really hot. But because it is seeping out of the sides I would think that a little too much was used or uneven spread of the paste and caused it to unevenly sets up and causes the excess heat grease to be forced out the sides. There are videos online that show you haw to take off extra compound and spread it evenly with a credit card.
Good luck and I hope to hear that this helped.