As with anything that depends on multiple things for its performance it makes up a part of it. All things being equal a drive with more cache will be slightly faster, but only when dealing with a full cache. The more cache a drive has, the more it can load what you think you'll need next, and the more it can send you at SATA speeds before it empties.
Its important to note that all things are not equal. A drive with a larger cache probably has other improvements as well such as higher aerial density and other improvements. This means your better off with the 32mb drive, probably.