today in my class my professor mentioned that

Cache misses becomes more expensive as the speed of the processor increases

But he didn't explain the reason. I searched this statement over the internet and found no answer whatsoever.

According to me, this statement is true because, when the speed of the processor increases it can execute more instructions at a given clock cycle, thus a miss leads to a stoppage of more instructions from executing. So cache misses becomes expensive as processor speed increases. Is my thinking correct or am I doing it all wrong?


Yes, that's correct. Assuming the speed of the cache doesn't change, a cache miss takes a fixed amount of time, and if the number of instructions per time unit increases, then more instructions get delayed because of the cache miss.

Modern processors try different methods to counteract this. Obviously you can use bigger caches which may lead to fewer cache misses. The disadvantage is that with the same technology, a bigger cache will run slower. Another method is organising the cache in such a way that "normal" code doesn't produce unneeded cache misses.

Another method is "streaming", where the cache figures out that you are reading from consecutive memory addresses, and uses this to start loading data before it is needed, reducing the average cost of a cache miss. That is often automatically. There may be instructions that load data from memory without caching it - if you process a gigabyte of data in consecutive memory, then caching any of it is pointless.

Then comes out-of-order execution: You may have one instruction waiting because of a cache miss, but other instructions not affected by this can go on executing. And with hyperthreading the processor executes two or more threads interleaving instructions; if thread A has to wait, then thread B continues at full speed.

| cite | improve this answer | |
  • $\begingroup$ Thank you... :) $\endgroup$ – Turing101 Sep 16 at 5:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.