[strike]Which bit? I see two heatsinks on there, and neither should get in the way.[/strike] Just saw the red oval... you do not want to take that off. It shouldn't get in the way of the GPU; the GPU goes to the left of the slot.
Have you actually tried fitting it in, or do you just think they'll get in the way?
In general, you don't want to be taking them off.