Yeah I'll try with some pliers. I was just scared of breaking it. Thanks for the info.
The plastic part of the connector is pretty tough. Use the pliers to grab that, not the wire. Try to rock the connector side to side as you pull directly away from the drive.
I've had some 4-pin Molex connectors that seemed to be welded to the drive, but some careful work with pliers saved the day.