I'm reading through Computer Architecture: A Quantitative Approach, Sixth Edition, and I'm trying to understand the issues surrounding tag storage in using HBM as an L4 cache. The primary concern mentioned in the book is "where do the tags reside?", since the sheer capacity (1 GiB) of the HBM with a small block size (64B) results in a large amount of tag bits (96MiB).

The book first suggests that the tag data might live outside of HBM, in a higher level of the memory hierarchy - except there is not enough SRAM capacity to store them, even in a modern L3 cache. It then goes on to suggest that keeping the tags in HBM as a solution to this. However, in L1-L3 SRAM caches, don't the tags reside in each of the caches themselves? Why would HBM be any different? Is it because we're somehow unable to perform hit detection/tag comparison on the HBM DRAM chips themselves (as is done in SRAM caches), so they need to be processed outside of HBM?

  • $\begingroup$ Please add the edition of the book: not all feature High Bandwidth Memory. And make the title stand out, or put it in quotes. $\endgroup$
    – greybeard
    Jun 12, 2020 at 9:53

1 Answer 1


Disclaimer: It's been a while since I last read that book. It was an edition that pre-dated HBM. So I can't really refer to the book to find out what it's saying.

The "level" of a cache refers to how many steps removed it is from the CPU's load/store hardware. It means nothing beyond that. You can't just say "L3" and assume that it refers to the same organisation.

In a modern machine, there is usually one L1 cache per core, split between instruction and data. It may be virtually addressed or physically addressed depending on the architecture.

L2 cache is typically unified, instructions and data. But is there one per core, or one per die?

And what does L3 cache mean? Is that the one shared between cores on a die? Is the GPU on the die, and the L3 is shared between the CPU cores and the GPU? Is the L3 cache off-chip? Or perhaps the tags are on-chip and the data is off-chip? Maybe this is a NUMA machine, and the L3 is shared between all processors on the same board?

I think the issue here may be that any signal that has to go off the chip is more expensive than one that stays entirely on the chip. It usually requires higher electrical currents, and more time since the tolerances of a circuit board aren't as tight as they are on a die. So if you can't fit an entire cache on a single chip or package, it can make sense to split the tags from the data.

If you are a CPU die, for example, and a different die invalidates a cache line (e.g. it changes state from Shared to Exclusive in the MESI model), then all levels of your cache need to be potentially invalidated. That is an operation that only requires accessing and modifying the tags, not the data. So keeping the tags on-chip and storing the data off-chip could make sense here. This was extremely common back when you couldn't pack quite so many transistors on a die.

I don't know if any of that helped.

  • $\begingroup$ That does actually help answer part of my question. If I understand correctly, accessing the cache can happen in three steps: 1. read tags corresponding to the address, 2. compare the tags to check for a hit, 3. read the data from the cache. So because reading the tags and reading the data are separate steps, they don't actually have to happen in the same physical part of the chip, although I'd imagine that might improve performance if you have the space. The remaining part of my question is why can't we check the tags on the HMB stack itself vs. sending to the CPU? $\endgroup$
    – nimiwio
    Jun 13, 2020 at 16:25
  • $\begingroup$ It's not just checking tags, it's also updating tags. Invalidating a cache line (e.g. because another CPU has modified it) doesn't require accessing the data, only the tags. $\endgroup$
    – Pseudonym
    Jun 16, 2020 at 2:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.