When in explorer, click tools/folder options. Make sure that "do not cache thumbnails" is unchecked.
If there are lots of files in the folder, it may take a while to display thumbnails.
Thanks Hawkeye22 for your help, but the solution you offered did not fix the problem.
FYI, The video thumbnails used to always appear. To fix them, I tried implementing things others had suggested too... but to no avail.