The answer to your first question is yes, all MP3 players can play podcasts. Podcasts are just MP3's. Beware that some podcasts are AAC encoded and most MP3 players (besides Apple) do not play AAC.
The answer to your second question is no. I haven't seen an MP3 player that allows you to use a USB jump drive as the storage medium. I think the reason for this is power requirements. Jump drives use 5V if I'm not mistaken; too high for a small MP3 device to handle. There are however plenty of MP3 players out there that are nothing more than jump drives w/ an MP3 decoder chip. Additionally, there are also MP3 players that use SD cards and memory stick cards as a storage medium for MP3's.