Ok I think I've targeted the process, when I'm using VoIP in Xfire, the process along with some audio driver process spikes the one core. I shut Xfire down and it seemed to take care of the issue. Weird!
I also updated my BIOS and installed some Intel IMF driver...dunno if that helped me at all or not. I also went into the BIOS and disabled the power saving option, and I think I'm getting a little better performance as well.
I think I can close this case for now, thanks for the replies everyone