Depends on the type of file, & what your "View" settings are for icons. For most picture files (i.e. JPEG), setting "View" to "Medium icons" or "Large icons" should show a small/medium preview of the picture itself.
For non-picture files, you really don't have a lot of options to have the icon show a preview of the file -- it's kind of hard to fit a bunch of text from an 8.5x11" page onto a 0.25x0.25" icon on the screen. However, single-clicking the file with the folder open will usually show a preview on the right side, so that works well for figuring out what regular files are for.