Wikipedia says that shared memory comes with lots of costs associated with cache coherence costs. But I thought the whole idea of shared memory is that all the CPUs access the same memory? So if one CPU changes that memory then other CPUs would access the same value? It would seem like this would require FEWER cache coherence costs? Is the idea that if one CPU changes its local cache before it writes to shared memory then other CPUs have to be notified?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
5
If two different processors share one memory, each having individual cache, they can end up having two different values in the same address.
Imagine each of two processors has private caches L1 and L2. The cache L3 is shared between both processors. Assume the processor A reads data from address X in L3 to L1 and the processor B reads the same data from the same address (address X in L3) to it's private cache L1. Then, if the processor A modifies the value and does a write-back, the processor B can't figure it out without the support of coherence protocol and would still have an old value in it's own cache.
Basically, you are right. The cache coherence protocol is a mechanism to notify processors about shared memory modification caused by other processors.
The main advantage of the shared memory architecture for a programmer is that there is no need to explicitly describe communication and interaction between processors (like you would do using MPI, for instance). The coherence and consistency of the memory is fully the responsibility of the hardware.
-
$\begingroup$ Arguably, the shared-memory model is an illusion at this point. You can program without making communication explicit, but the cost of communication is still there, however implicit. Some say cache coherence protocols are major hurdles for efficient parallel algorithms; see e.g. false sharing. $\endgroup$– RaphaelCommented Sep 10, 2013 at 10:31
-
1$\begingroup$ Memory consistency is also a burden for the programmer. Although cache coherence is usually the hardware job and ensures that all writes to a given location are eventually seen and seen in the same order by all processors, you have to pay attention when several memory locations are involved. If you don't you may have a thread which writes A and B in that order, another thread which reads B and A in that order, and yet see the new value of B and the old value of A. $\endgroup$ Commented Sep 10, 2013 at 12:11
-
$\begingroup$ Thank you for the clarification. Coherence protocols do not exempt the programmer from preventing race conditions and possible memory conflicts. But. Coherence protocols ensure consistency in terms of reads/writes with respect to other memory locations. It means that a write is not considered as complete until other processors are aware of the effect. Also the order of processor writes can't be changed. So all writes to the memory are preserved by the instruction order in the application. $\endgroup$ Commented Sep 10, 2013 at 13:17
-
$\begingroup$ You have claimed that
The coherence and consistency of the memory is fully the responsibility of the hardware. However, what are the differences between cache coherence and memory consistency? Or, are they the same? $\endgroup$– hengxinCommented Dec 18, 2013 at 7:04 -
$\begingroup$ The main difference is that cache coherence defines the behaviour with respect to the same memory address (i.e. the memory access model is coherent if a processor is able to read the recent value from the memory only), whereas memory consistency model specifies the contract between different processors regarding different memory location accesses (i.e. in what order processors must read values from the memory written by other processors? What happens if several processors write values to different locations simultaneously with respect to reads?). $\endgroup$ Commented Dec 18, 2013 at 12:37
$\begingroup$
$\endgroup$
You are correct: when one processor changes a memory location that is locally cached, then all other processors that are sharing that memory location need to be notified.
So why have local caches? If you didn't then every memory access would be as slow. You want most memory accesses to be fast, so each processor should have its own fast cache. This works extremely well for data that is used by only a single thread of computation (for example: each thread has a private stack, and no other thread will read or write that data.) Likewise it works extremely well for read-mostly data. For example, data structures that get initialized once and then read by many threads. The code of the program is also read only, so does well being cached.
For the few memory locations that really are shared, you need to keep the caches coherent. This involves detecting that a memory location is shared, and then notifying any of the other caches that are currently caching that memory location. You have to pay the cost of the book-keeping to detect which memory locations are shared even if you never actually use the shared memory.
answered Sep 10, 2013 at 2:13