XMP doesn't overclock a system from the typical definition of what an overclock is. If you're talking about the little boost that XMP typically provides, either by nudging the BCLK or by adjusting the way Intel turbo boost works, I'm afraid you have to disable XMP to get rid of those. The two go together, hand in hand.