1
$\begingroup$

From what I understand, in a pipelined CPU, every stage takes 1 cycle. But instructions are fetched from memory which takes up to ~150 cycles. The CPU fetches most instructions from the L1-cache, but I've read that it takes around 4 cycles. From that logic, a new instruction should only start every 4th cycle, which obviously makes no sense? enter image description here This diagram shows a new instruction starts every cycle.

So, how does the CPU fetch a new instruction every cycle, if it takes more than one cycle to even fetch an instruction?

Improve this question
$\endgroup$
4
  • $\begingroup$ It probably fetches a bunch of instructions at once $\endgroup$
    – nir shahar
    Commented Sep 15, 2021 at 15:10
  • $\begingroup$ Yeah, I figured. Where are they then stored? $\endgroup$
    – seb
    Commented Sep 15, 2021 at 16:30
  • 1
    $\begingroup$ Somewhere on the chip :-) $\endgroup$
    – rici
    Commented Sep 15, 2021 at 19:55
  • $\begingroup$ Assume company A builds a processor that fetches an instruction from L1 cache every four cycles. And your company B offers you a million dollars to read an instruction every cycle. So what do you do? PS My computer reads up to 4 instructions per cycle per core. It has 8 cores so that's 32 instructions per cycle. $\endgroup$
    – gnasher729
    Commented Oct 17 at 17:19

2 Answers 2

Reset to default
3
$\begingroup$

So, how does the CPU fetch a new instruction every cycle, if it takes more than one cycle to even fetch an instruction?

It could be done by starting the next fetch before the previous one completed.

An L1 hit may "take around 4 cycles", but the number that is thrown around a lot is the load-use latency of hitting the L1D cache and then using the result in an ALU operation (which isn't representative nor predictive of what goes on in the front-end of a processor), and it normally refers to architectures that have a significantly longer pipeline the one that the diagram in your question is for. If the diagram shows a single cycle FI stage, I'm inclined to believe it: that architecture probably fetches an instruction in a single cycle (which was, and still is, a normal thing for architectures that aren't "super-pipelined"), otherwise the diagram should show more fetch stages.

The kind of architecture which that "4 cycles" figure is about, look more like this (actually the load-use latency is only 3 cycles here, but you get the idea):

AMD bobcat pipeline

How long the I-cache access takes is not so clear from this diagram, but I argue that it should be at most 3 cycles and possibly less, because after 3 cycles there is an arrow going down to the decoders and by that time the instruction data should be available, otherwise there would be nothing to decode. But how long the cache access takes is not that important (well it is important, but it's not as bad as limiting fetches to on in k cycles where k is the cache latency), whether it takes 1 or 2 or 3 cycles, one access can happen every cycle, so when all goes well the pipeline can stay full.

Improve this answer
$\endgroup$
0
$\begingroup$

The instructions are loaded into the on-chip Level 1 (L1) instruction cache, so that they can be loaded every cycle. Level 1-3 (L1-L3) cache subsystems are typically implemented with static random-access memory (SRAM) subsystems to enable this.

Superscalar processors allow multiple instructions to be fetched per clock cycle.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.