This can damage your speakers. I wouldn't lose sleep over the times it has happened thus far, but I would make it a point to avoid it going forward.
Try splitting the signal before your computer speakers so that your sound system gets it's audio from your computer directly. Then turn the volume on your computer speakers down so that Skype doesn't blair through.
Just a random example of the splitter I'm talking about.http://www.staples.com/Staples-Headphone-Splitter/product_837541