If it is an IDE harddrive it is a very bad idea. Nothing might happen or you could fry the drive,controller on motherrboard,or motherboard itself.
Well, I assumed it was an external HDD, while you assumed it was an internal one. The op did not specify, but given the fact that removing an internal one requires loosening some screws and unplugging some power and data cables, I would say it is unlikely to happen by accident. Intentionally... why would anyone do that?
On the other hand, an external drive is pretty easy to unplug, hanging from a USB port, and someone who's not aware of the format operation going on might actually do that.
I guess we'll not know until the op specifies which is the case...