4
$\begingroup$

What is the trend over time for the size of registers of x86 CPUs? I know that we have today 512-bit registers, but is there a kind of Moore's Law to predict the size of registers in the future?

Thank you very much.

$\endgroup$
5
$\begingroup$

Moore's law is only indirectly involved here. NVidia's SIMD registers are 1024 bits wide, have been that wide since at least 2006, and haven't grown. Intel's slowly grew from 64-bit MMX in 1996, to 128-bit SSE in 1999, to 256-bit AVX in 2008, to the 512-bit AVX-512 in 2015.

My prediction is that neither Intel nor NVidia will grow their SIMD registers any wider from here on out. The cost would be large, and the benefits decrease with each doubling.

The reason the cost would be large for Intel is that Intel has a whole bunch of inter-related design decisions that would need to be revisited. Intel processors have used a cache-line width of 512 bits since the Pentium-4. The cache ports to the cpu pipeline are 512-bits wide. The BTB designs probably assume 64-byte cache lines. I assume the coherence busses for multi-socket systems are 512-bits wide. (I'm in the software division, not the hardware division.) There is probably quite a bit of legacy code that has now been optimized for 512-bit cache lines. For example: to avoid false-sharing overhead on cache lines, data structures may be aligned on 64-byte boundaries. Customers would get annoyed if they spent thousands of dollars for a new server only to discover that their legacy code now runs slower. I would guess that NVidia faces similar constraints.

The benefits decrease with each doubling because of something similar to Amdahl's law. The benefit of SIMD is supposed to be that you double your ALU bandwidth, while keeping the area (and power) of the rest of your pipeline constant. Suppose your processor is 16mm2, of which 1mm2 is your 32-bit ALU and register file, and you are getting 1 GFlOPS. That's 1/16 GFlOPS/mm2. Move to 64-bit SIMD, and your performance doubles, but so does the size of your ALUs and register file. So now you are getting (at best) 2 GFloPS/17mm2, which is almost twice as good. But when you get up to 512-bit SIMD you have 16 GFlOPS/31mm2. Now doubling to 1024-bit SIMD gets you to 32 GFlOPS/47mm2, which is only 32% better.

$\endgroup$
  • $\begingroup$ Vectors can also reduce operation tracking overhead. The size of the vector does not have to equal the width of per cycle execution. In addition, with the addition of scatter-gather (or even just non-unit stride), cache line size becomes less important. $\endgroup$ – Paul A. Clayton May 8 '16 at 19:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.