Why are they called "memory sticks" or "flash memory"...

Well, it looks like a stick of gum, but "stick of memory" sounds stupid, so memory stick stuck.

As for flash drives, they use flash memory. They are also not a storage device that you put into another device attached to the computer (ie. CD into CD drive attached to computer or disk into floppy drive attached to computer). They are their own drive attached via USB.

First off Memory Stick is a proprietary trademark of Sony, There are many flash based storage devices all with different connectors and different names CF, MMC, SD and so on. Flash drives comes from disk drives or hard drives it's just another form of storage.

"Flash drive" has nothing to do with USB.
USB is just a connector type that connects the Flash Memory to the PC.
Search for Flash memory on Wiki and you'll see a photo of a "Thumb Drive" that uses an SD connection.

There are many terms for such drives, many of which are proprietary,
such as Thumb Drive (Trek Technologies), DiskOnKey (IBM), Jump Drive (Lenovo), SanDisk Cruzer, et al.
The term "Flash Drive" is more generic to reflect the flash memory that it uses.

Flash Memory gets its name from the type of memory that it is.
Flash is a sub-type of a form of memory chip called an EEPROM.
EEPROM is Read-Only memory chip that can be erased and programmed electronically.
And by programming, they mean "re-write stuff on it".
So even though it's called "Read-Only" memory, you can also write to it.

EEPROM is an improvement over its predecessor, EPROM.
EPROM can also be erased and re-written to, but not electronically.
To erase an EPROM, you have to shine a very bright UV light on it.
That sets all of the bits to 0.
An EPROM chip is contained in a case that has a little window on it,
and you litterally FLASH the chip with the UV light to erase it.

From this comes the term "Flashing the memory" to mean that you are erasing the chip,
or more precisely, setting all the bits to 0.

When EEPROM was later invented, the UV light was no longer necessary
because the chip could be erased with electrical current through circuits.
But term "Flash the memory" stuck,
keeping it's previous meaning of setting all of the bit values to 0 on the entire chip.

As chip technology evolved, users were able to selectively erase and write to specific bits on the memory, but again, the term "Flash Memory" stuck.

So the name actually comes from flashing UV light onto the chip to allow you to reuse it.