It depends on the OS and interface.
It may be possible with Windows via an eSATA external HDD, but not via USB. For numerous reasons, mainly being Windows does not support it and least of all, speed. USB 3.0/3.1 is certainly faster than 2.0, but nowhere near as fast as SATA3. eSATA may be possible, but drives availability is limited, at best.
Certain Linux distros can boot from external HDDs, so I would assume it's theoretically possible - but I don't know for sure.
I'd question why you'd want to do it though? Assuming you're talking Windows, there's a whole lot of potential driver/chipset issues (and licensing issues) if you intended to use it on multiple systems. If you just want it to protect data etc, you don't need the OS on the drive, just the files.