Gently take it off of what, the socket? I would use the toothpick to gently pick out any clumps between the fingers and then a coffee filter and rubbing alcohol to gently rub the fingers as per above. Don't bend the fingers.
Is the gunk only on the CPU like in the middle where the caps are? Or is it on the board and the pins (or arms/fingers) in the socket as well? Either way, just be careful not to bend anything or break anything off. Gentle and rubbing alcohol is the only thing you need to know.
Ok, I have a similar situation. I have a little bit of thermal paste that kinda drapped across the bottom of the CPU (its a line a little bit thicker than a couple of hairs that are on the very tips of the cpu pins). So, I have a CPU, q-tip(s), rubbing alcohol and coffee filter(s). I wet the q-tip with rubbing alcohol and apply the q-tip (gently) to the areas on the CPU that has thermal paste on it. I don't understand what role the coffee filter plays.
Perhaps a picture would help us help you? I generally dislike submerging my CPU's in anything.
I know some thermal pastes are non-conductive so I would wonder if this is the problem at all. Obviously there is no need for thermal paste to be on the underside but if the computer worked at one time that "goo" was probably not the reason for the POST problem. Since it doesn't move around the pins it is unlikely that the just shifted at some time and shorted things out resulting in your POST problem.
Who knows, perhaps it is some goo meant to keep the contacts from corroding (not that they would) while still allowing contact between them and the socket.
I guess what I'm saying is that clean it off as best you can but I would think about other options that could be causing the problem because I don't think this goo is it.