# Confusion regarding calculation of estimated memory access time in a system containing only a cache and main memory for simplicity

Let us consider a system having cache and main memory. Now suppose we are asked to find the average memory access time. Let $$h$$ be the hit ratio for the cache, $$t_c$$ be the cache access time, $$t_m$$ be the time to access a word of data from the main memory and cache block size be $$b$$ words.

The picture below is from William Stalling's Computer Organization and Architecture text.

From the diagram above, it seems that in case of a cache miss, the transfer of the block of words from main memory to cache takes place at the same time the word required by CPU is transferred to it directly by main memory.

In most places I have seen unclear formula for Estimated Memory Access Time $$\text{(EMAT)}$$ as:

$$\require{color}\colorbox{cyan}{\text{EMAT}= t_c+(1-h)* \text{main-memory-access-time}}$$

But they do not say clearly whether this $$\text{main-memory-access-time}$$ is for a single word or for a block of words.

What I feel from the block diagram shown above:

$$\require{color}\colorbox{cyan}{\text{EMAT}= t_c+(1-h)*t_m}$$

Since, once the required word has reached the CPU there ends the memory access as seen by CPU while in the background the block transfer might occur (taking a total of $$b*t_m$$ time for the block transfer)... So in this point of view $$\text{miss-penalty} = \text{memory-access-time-for-one-word}$$.

Upon second thoughts the thing which bothers is that, after having a miss, since the block transfer occurs in background, during that $$[(b*t_m) + t_c]$$ time, reference to the cache block which is being transferred shall lead to a miss, so this makes me feel that the $$\text{EMAT}$$ could have been:

$$\require{color}\colorbox{cyan}{\text{EMAT}=t_c+(1-h)*[(b*t_m) + t_c]}$$

So, could anyone guide me as to how I should proceed with such calculations?

Think of RAM access like accessing one train wagon, when you have a locomotive that can pull a dozen wagons. Getting one wagon or 12 wagons takes about the same time.

Same with RAM. The time to read from memory consists of the time for sending an address to RAM, plus the time for the RAM to find the line of memory containing the address, and the time of transmitting the data. The transmission is done several words at a time. Trying to read a single word would safe very little because the RAM would have to find that word in the memory line, wastes all the extra words that could be transmitted at the same time.

There are systems where a cache line needs two transmissions from RAM, and the RAM sends the half cache line containing the requested memory first. Still, two transmissions take very little longer than one.

So when the memory word that you wanted arrives at the CPU, the cache line is filled or will be filled very quickly.

• @gnasher725 Thanks for your answer. I was having one doubt. The transmission is done several words at a time. But the data bus size is the word size of the system, so at a time only 1 word could be delivered by RAM. But I guess you are pointing to the concept that a considerable chunk of words are read from the main memory into the row buffers of the controller. Then very fast these words (in the buffer) can be delivered via the data bus. Is this something like that? Aug 28 at 7:35
• See for example en.wikipedia.org/wiki/DDR4_SDRAM which says that on DDR4 RAM, eight 64-bit words are transferred at a time. If transferring single words were possible, it wouldn't be faster than eight 64-bit words. Even if you need 16 64-bit words for a 1024bit = 128 byte cache line, that doesn't take much longer than eight 64-bit words, because the address is transmitted only once, and the memory line is located only once. Aug 28 at 22:19
• Thanks, with numerical example things got more clear to me. Aug 29 at 13:59