Confusion regarding calculation of estimated memory access time in a system containing only a cache and main memory for simplicity

Question

Let us consider a system having cache and main memory. Now suppose we are asked to find the average memory access time. Let $h$ be the hit ratio for the cache, $t_c$ be the cache access time, $t_m$ be the time to access a word of data from the main memory and cache block size be $b$ words.

The picture below is from William Stalling's Computer Organization and Architecture text.

RA: Required Address

From the diagram above, it seems that in case of a cache miss, the transfer of the block of words from main memory to cache takes place at the same time the word required by CPU is transferred to it directly by main memory.

In most places I have seen unclear formula for Estimated Memory Access Time $\text{(EMAT)}$ as:

$$\require{color}\colorbox{cyan}{$\text{EMAT}= t_c+(1-h)* \text{main-memory-access-time}$}$$

But they do not say clearly whether this $\text{main-memory-access-time}$ is for a single word or for a block of words.

What I feel from the block diagram shown above:

$$\require{color}\colorbox{cyan}{$\text{EMAT}= t_c+(1-h)*t_m$}$$

Since, once the required word has reached the CPU there ends the memory access as seen by CPU while in the background the block transfer might occur (taking a total of $b*t_m$ time for the block transfer)... So in this point of view $\text{miss-penalty} = \text{memory-access-time-for-one-word}$.

Upon second thoughts the thing which bothers is that, after having a miss, since the block transfer occurs in background, during that $[(b*t_m) + t_c]$ time, reference to the cache block which is being transferred shall lead to a miss, so this makes me feel that the $\text{EMAT}$ could have been:

$$\require{color}\colorbox{cyan}{$\text{EMAT}=t_c+(1-h)*[(b*t_m) + t_c]$}$$

So, could anyone guide me as to how I should proceed with such calculations?

Answer 1

Think of RAM access like accessing one train wagon, when you have a locomotive that can pull a dozen wagons. Getting one wagon or 12 wagons takes about the same time.

Same with RAM. The time to read from memory consists of the time for sending an address to RAM, plus the time for the RAM to find the line of memory containing the address, and the time of transmitting the data. The transmission is done several words at a time. Trying to read a single word would safe very little because the RAM would have to find that word in the memory line, wastes all the extra words that could be transmitted at the same time.

There are systems where a cache line needs two transmissions from RAM, and the RAM sends the half cache line containing the requested memory first. Still, two transmissions take very little longer than one.

So when the memory word that you wanted arrives at the CPU, the cache line is filled or will be filled very quickly.