Caching in Multi-Processor Systems

The improvement in computer performance is currently achieved by adding more processors to the system. Today, it actually means integrating multiple cores to one chip. The need for caching still exists. The data must be close to processor for efficient processing. With increased number of cores on the chip the cache organization within the chip is also important. Several alternatives for private or shared cache organization have been proposed and evaluated in the literature. Common organization is that the L1 caches are private and the L3 cache is shared, but the interesting issue is the L2 cache. There multiple alternatives have been proposed.

Sharing the cache by all cores makes the cache coherence issue easier, since there is just one copy of the memory content in the cache. However, increasing the cache size quite easily means that different cache lines have different latencies in the core access. Some parts of the cache are further away from some cores than other parts. Also the cores are competing for the cache access and may occasionally need to wait for another core.

Splitting the cache area for private caches reduces the access latencies and removes the access competition, but introduces the problem of cache coherence. Now multiple private caches may contain the same cache line and the write on any core must invalidate the line on other caches. An interesting approach is the cooperative caching, where a directory structure is added to handle the coherence issues and also to avoid necessary memory access, if the data is on another core's cache already.

The issue of shared versus private cache organization is not just black and white. With several cores on the chip, it is possible to organize the cache is such a way that a pair of cores share cache, but other core pairs are not allowed to access this cache.