Quick answer: Yes. That's how all my backups are done. I have two enclosures that can hot-plug and hot-remove (as long as I remember to Safely Remove the media) drives, and this is where my backups, and offline storage, live. Shameless plug: http://kingwin.com/products/cate/mobile/racks/kf_1000_bk.asp
That said, two warnings: Whether or not hot-swap is enabled depends on the motherboard / controller and the OS. On my Asus P7P55D Deluxe, only one of the ports was hot-swappable under XP; they all are under Windows 7.
The second warning: Backups are critical for your business, and bare drives are vulnerable to static, falls, liquids, and large dogs. Have good, antistatic, secure storage for the removed drives. Make copies and send them off-site if your business depends on them - they won't do you any good if the roof of your datacenter collapsed from the snow.
Edit: I just re-read your post to make sure that I was on topic. You do take backups offsite. Good for you.