Cache can best be descrived as REALLY fast memory with VERY fast latencies (almost instant, especially the L1 cache).
Each level of cache on a CPU is slightly slower then the one before it. The fastest access comes from the L1 cache, but this is very limited in size. The L2 cache is bigger, but slightly slower then the L1 cache. Likewise, the L3 cache is the biggest yet, but is slower to access then the L2 cache.
In some CPU's, the L1/L2 cache is unique for each individual core [each core gets its own cache], and the L3 cache is a global pool all the cores can access (simmilar to normal RAM, just with faster access times really).
As such, a larger cache will to some degree increase performance, as a larger cache can hold more data that can be more quickly accessed by the CPU.
Theres a LOT more detail of course, but this is the easiest way to explain how the CPU cache works without going into specifics.http://en.wikipedia.org/wiki/CPU_cache
Specifically, read up on the K8 cache architecture on this page for a more specific example: