17
$\begingroup$

I thought this question is better served in the CS part of Stack Exchange. Now that we have GPGPUs with languages like CUDA and OpenCL, do the multimedia SIMD extensions (SSE/AVX/NEON) still serve a purpose?

I read an article recently about how SSE instructions could be used to accelerate sorting networks. I thought this was pretty neat but when I told my comp arch professor he laughed and said that running similar code on a GPU would destroy the SIMD version. I don't doubt this because SSE is very simple and GPUs are large highly-complex accelerators with a lot more parallelism, but it got me thinking, are there many scenarios where the multimedia SIMD extensions are more useful than using a GPU?

If GPGPUs make SIMD redundant, why would Intel be increasing their SIMD support? SSE was 128 bits, now it's 256 bits with AVX and next year it will be 512 bits. If GPGPUs are better processing code with data parallelism why is Intel pushing these SIMD extensions? They might be able to put the equivalent resources (research and area) into a larger cache and branch predictor thus improving serial performance.

Why use SIMD instead of GPGPUs?

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ It is a challenge in itself to feed the GPU with enough data to keep it busy. Data transfer between the host and the device is practically always the bottleneck. Certain operations are better supported on CPUs (e.g. carry-free multiplication; see PCLMULQDQ). $\endgroup$
    – Juho
    Commented Sep 2, 2014 at 20:10
  • 2
    $\begingroup$ @Juho Don't new devices like AMD's APUs have the GPU and CPU on the same die? Does this eliminate the bottleneck? $\endgroup$
    – jonfrazen
    Commented Sep 2, 2014 at 20:17
  • 1
    $\begingroup$ When all is said and done, a vector instruction is still a single instruction, and the cost to schedule and execute it is the same as any other single instruction. It only makes sense to run jobs on the GPU when the benefit outweighs the cost. Also consider that you get one SIMD unit per core, but typically only one GPU per chassis, and the GPU is at the moment a shared resource. This limits the number of jobs that you can run on a GPU. (The number of cores is increasing all the time, but the number of GPUs is not.) $\endgroup$
    – Pseudonym ♦
    Commented Sep 3, 2014 at 0:26
  • $\begingroup$ Intel does not do much in terms of GPUs (apart from Larrabee/Knights Landing :) ), so I guess it is natural for them to try to push AVX instead. Although heavy AVX use may be very performant, it now results in downclocking on their newer CPUs, so they may be hitting limits with something. $\endgroup$
    – nsandersen
    Commented Jul 20, 2017 at 11:38

1 Answer 1

Reset to default
19
$\begingroup$

Nothing is free. GPGPUs are SIMD. The SIMD instructions on GPGPUs tend to be wider than the SIMD instructions on CPUs. GPGPUs tend to be fine-grained multi-threaded (and have many more hardware contexts than CPUs). GPGPUs are optimized for streaming. They tend to devote a greater percentage of area to floating point units, a lower percentage of area to cache, and a lower percentage of area to integer performance.

Let's do a comparison. Intel's core i7-5960x has 8 cores, each with 4-wide (double precision) SIMD, running at 3 GHz (3.5GHz turbo), a 20M L3 cache, consumes 356mm^2 and 140W and costs \$1000. So 8*4*3*4 = 384 double precision GFlops. (The extra 4x is because you can do two fused-multiply-adds per vector lane per cycle.) It can do 768 single precision GFlops. That's about 1.08 DP GFlops/mm^2 and 2.75 DP GFlops/Watt. There's also about 57.5 KB/mm^2 of on-chip cache.

NVidia's GeForce GTX Titan Black has 15 SMXs, each with 32-wide double precision SIMD, running at 890MHz (980MHz turbo), 3.5M of L1+L2 cache, consumes 561mm^2, 250W and costs \$1000. So 15*32*.89*4 = 1709 double precision GFlops. (Same 4x from two fused-multiply-adds per vector lane per cycle.) It can do 5126 single precision GFlops. That's about 3.05 DP GFlops/mm^2 and 6.8 DP GFlops/Watt. So 3x the DP floating point density per unit area and 3x the DP floating point power efficiency. And the tradeoff? 6.4 KB/mm^2 of on-chip cache. About 9x less dense than the CPU.

So the main difference is that the GPU has chosen an area balance that strongly favors floating point (and especially single-precision floating point) over cache. Even ignoring the fact that you need to copy stuff back and forth between the CPU and GPU to do I/O, how well the GPU is going to do compared to the CPU depends on the program you are running.

If you have a data parallel floating point program with very little control divergence (all the vector lanes are doing the same thing at the same time) and your program is streaming (can not benefit from caching), then the GPU is going to be about 3x more efficient per unit area or per Watt. But if you have any significant amount of divergent control, non-data-parallel work to do, or could benefit from large read-many-times data structures, the CPU will probably do better.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.