Here's an example: you have a hard drive with 1 TB of space on it, which can be accessed at a rate of 100 MB/s. You also have 64 MB of RAM, included inside the hard drive, which can be accessed at much higher speeds, let's say 60 times faster.
Now, let's say you're a programmer and you're writing some code to read a 20 MB JPG file from the hard drive and display it.
Without the cache, the second time you access that file is just as fast as the first time, because there's nothing different between access #1 and access #2.
With the cache, the first time you access the file is still slow, but the file is copied into the cache. The second time, it is taken from the cache, and that takes 60 times less time.
The "transparent" part means you, as the programmer who writes the code to read the file, don't even need to know if there is a cache there or how large it is or what type it is or anything else. You do not write any special code to deal with the cache. Your code is the same whether the cache is there or not. The cache sits between your program and the hard drive but doesn't require any changes in your program. Think of it as your code looking at the hard drive and not seeing the cache even though it is right there in the middle.
While looking up basic terminology for CPUs, I came across this phrase from Wikipedia (I began on the CPU cache page, and then ended up on the 'cache' in computer terms page).
I had a basic idea of what caches do in general, but I wanted to be sure I was right and see if I missed anything.
So.. I ended up on the page for cache, in terms of computer science, and it had this phrase:
'In computer science, a cache is a component that transparently stores data so that future requests for that data can be served faster.'
I understand that whole sentence with no problems, excluding the bit about transparently storing data.
It might be my OCD or my 'but I must know that which I do not' quirk.. but I'd really like to know what this phrase means.
Thanks so much!
It means that the actions of the cache acting on some data store are invisible to the mechanism requesting data from that data store.
Caches by design exploit differences in performance characteristics in order to improve some other performance characteristic. The most commonly exploited performance characteristic (as pertains to caching) is response time. In addition to this, two other relationships are also exploited, those of space (spatial) and time (temporal).
The spatial relationship says that data which is adjacent to data that has been accessed is more likely to be accessed than data that is not.
The temporal relationship says that data which has been accessed is more likely to be accessed again than data that has not.
Thus, a fast but low capacity cache will take frequently accessed data, as well as data surrounding previously accessed data from a slow but high capacity memory device and store it itself. When a device makes a request for some data from that memory device, the cache will receive the request first. If the data requested is in the cache, it will be returned very quickly (in computer engineering parlance, this is called a cache hit). If the data requested is not in the cache, then the request is passed to the memory device where it is returned as normal (in computer engineering parlance, this is called a cache miss).
Although the device requesting data from the memory device must be aware that the data may not be returned in constant time (otherwise that would defeat the purpose of the cache), it need not know the size of the cache, the scope of the cache, or even if a cache is present. Similarly, the memory device on the far side remains blissfully unaware that there's another, faster, smaller memory device sitting in front of it.
Since you were researching CPU caching, lets talk about that.
When modern CPUs execute instructions they do so through a process known as pipelining. One of the first stages of an instruction pipeline is to send an address to the memory to fetch an instruction. On slow microcontrollers (25-50Mhz instruction clock) with embedded ROMs (4KiB-64KiB), the memory will respond with the data at that address before the microprocessor has started the next clock cycle. In this case, there's no need for caching. However, on modern high frequency CPUs (3-4Ghz) working with large amounts of DRAM and memory mapped IO, it can take several hundred CPU cycles before the DRAM responds with the first instruction. Thus, the CPU instruction pipeline does not access the DRAM directly when looking for instructions, rather it accesses a cache called the Level 1 instruction cache.
On Intel microprocessors, the L1 cache is broken into a 32KiB instruction cache and a 32KiB data cache. This is important because data is accessed later on in the pipeline. The purpose of the L1 instruction and data caches is to minimize the access time. Beyond this there's an L2 cache dedicated to each core, right now that's a standard 256KiB. The L2 cache is larger, but slower than the L1 cache. If the desired instruction or data word is not found in the L1 cache, then it checks the L2 cache. Beyond that, there's an L3 cache which is shared by all instruction cores in the microprocessor. Some AMD microprocessors lack an L3 cache. Intel and AMD have designed their cache controllers so well that it's actually quite rare for a desired instruction or piece of data to be completely absent from the CPU's cache architecture. So, rather than waiting 300 cycles for an instruction from the DRAM to arrive, the pipeline need only wait 4 cycles (level 1 hit), 11 cycles (level 2 hit), or ~30 cycles (level 3 hit).
Despite all of this, the individual x86 CPU instructions do not care whether or not a cache is present. They operate either on registers (no cache delay), or memory addresses (transparently).
Thanks so much for the detailed explanations, aevm and Pinhedd!! Thank you also, Pinhedd, for linking it to CPU information! I would have replied earlier but I was up at our family cottage and there was no internet access.