Michelle Beth :
Hello Tom's Community,
While looking up basic terminology for CPUs, I came across this phrase from Wikipedia (I began on the CPU cache page, and then ended up on the 'cache' in computer terms page).
I had a basic idea of what caches do in general, but I wanted to be sure I was right and see if I missed anything.
So.. I ended up on the page for cache, in terms of computer science, and it had this phrase:
'In computer science, a cache is a component that transparently stores data so that future requests for that data can be served faster.'
I understand that whole sentence with no problems, excluding the bit about transparently storing data.
It might be my OCD or my 'but I must know that which I do not' quirk.. but I'd really like to know what this phrase means.
Thanks so much!
-Michelle
It means that the actions of the cache acting on some data store are invisible to the mechanism requesting data from that data store.
Caches by design exploit differences in performance characteristics in order to improve some other performance characteristic. The most commonly exploited performance characteristic (as pertains to caching) is response time. In addition to this, two other relationships are also exploited, those of space (spatial) and time (temporal).
The spatial relationship says that data which is adjacent to data that has been accessed is more likely to be accessed than data that is not.
The temporal relationship says that data which has been accessed is more likely to be accessed again than data that has not.
Thus, a fast but low capacity cache will take frequently accessed data, as well as data surrounding previously accessed data from a slow but high capacity memory device and store it itself. When a device makes a request for some data from that memory device, the cache will receive the request first. If the data requested is in the cache, it will be returned very quickly (in computer engineering parlance, this is called a
cache hit). If the data requested is not in the cache, then the request is passed to the memory device where it is returned as normal (in computer engineering parlance, this is called a
cache miss).
Although the device requesting data from the memory device must be aware that the data may not be returned in constant time (otherwise that would defeat the purpose of the cache), it need not know the size of the cache, the scope of the cache, or even if a cache is present. Similarly, the memory device on the far side remains blissfully unaware that there's another, faster, smaller memory device sitting in front of it.
Since you were researching CPU caching, lets talk about that.
When modern CPUs execute instructions they do so through a process known as
pipelining. One of the first stages of an instruction pipeline is to send an address to the memory to fetch an instruction. On slow microcontrollers (25-50Mhz instruction clock) with embedded ROMs (4KiB-64KiB), the memory will respond with the data at that address before the microprocessor has started the next clock cycle. In this case, there's no need for caching. However, on modern high frequency CPUs (3-4Ghz) working with large amounts of DRAM and memory mapped IO, it can take several hundred CPU cycles before the DRAM responds with the first instruction. Thus, the CPU instruction pipeline does not access the DRAM directly when looking for instructions, rather it accesses a cache called the Level 1 instruction cache.
On Intel microprocessors, the L1 cache is broken into a 32KiB instruction cache and a 32KiB data cache. This is important because data is accessed later on in the pipeline. The purpose of the L1 instruction and data caches is to minimize the access time. Beyond this there's an L2 cache dedicated to each core, right now that's a standard 256KiB. The L2 cache is larger, but slower than the L1 cache. If the desired instruction or data word is not found in the L1 cache, then it checks the L2 cache. Beyond that, there's an L3 cache which is shared by all instruction cores in the microprocessor. Some AMD microprocessors lack an L3 cache. Intel and AMD have designed their cache controllers so well that it's actually quite rare for a desired instruction or piece of data to be completely absent from the CPU's cache architecture. So, rather than waiting 300 cycles for an instruction from the DRAM to arrive, the pipeline need only wait 4 cycles (level 1 hit), 11 cycles (level 2 hit), or ~30 cycles (level 3 hit).
Despite all of this, the individual x86 CPU instructions do not care whether or not a cache is present. They operate either on registers (no cache delay), or memory addresses (transparently).
I hope that answered your question.