OK, analogy time here.
Your HDD is a bookshelf, and it is full of unmarked books. When you write data, your writing inside these unmarked books. Formatting the HDD is the equivalent of writing a guide to where all the information is, because just looking at the bookcase you cant tell where anything is. This is why if you reformat a drive you "lose" all your data on it, because you just destroyed your guide to where all the information is. Data recovery software can still read whats written there (the data is still written in the "books" and can be recovered).
This needs to happen because quite simply your OS needs to know where data is on the drive, and to search the whole drive linearly for it would be slow as all hell (especially considering the data could be fragmented across the drive).
turkey3_scratch :
For example, why could one not just copy the entire C drive to another hard drive and have two hard drives of Windows
You can absolutely do this, no reason you cant.
You will never have two C: drives on the one system, since "C: Drive" is reserved only for the partition the OS is running off, however there is no reason you cant have an identical copy of C: in the same system (it would just come under a different drive letter).
turkey3_scratch :
Are USB Drives formatted and how does that differ from hard drives?
All storage devices have to be formatted. USB drives and other small-capacity flash memory are typically formatted by a different standard (typically FAT32) thats more suited to their technologies than NTFS (typical HDD format in Windows).