# Why use SIMD if we have GPGPU?

I thought this question is better served in the CS part of Stack Exchange. Now that we have GPGPUs with languages like CUDA and OpenCL, do the multimedia SIMD extensions (SSE/AVX/NEON) still serve a purpose?

I read an article recently about how SSE instructions could be used to accelerate sorting networks. I thought this was pretty neat but when I told my comp arch professor he laughed and said that running similar code on a GPU would destroy the SIMD version. I don't doubt this because SSE is very simple and GPUs are large highly-complex accelerators with a lot more parallelism, but it got me thinking, are there many scenarios where the multimedia SIMD extensions are more useful than using a GPU?

If GPGPUs make SIMD redundant, why would Intel be increasing their SIMD support? SSE was 128 bits, now it's 256 bits with AVX and next year it will be 512 bits. If GPGPUs are better processing code with data parallelism why is Intel pushing these SIMD extensions? They might be able to put the equivalent resources (research and area) into a larger cache and branch predictor thus improving serial performance.

Why use SIMD instead of GPGPUs?

• It is a challenge in itself to feed the GPU with enough data to keep it busy. Data transfer between the host and the device is practically always the bottleneck. Certain operations are better supported on CPUs (e.g. carry-free multiplication; see PCLMULQDQ). – Juho Sep 2 '14 at 20:10
• @Juho Don't new devices like AMD's APUs have the GPU and CPU on the same die? Does this eliminate the bottleneck? – jonfrazen Sep 2 '14 at 20:17
• When all is said and done, a vector instruction is still a single instruction, and the cost to schedule and execute it is the same as any other single instruction. It only makes sense to run jobs on the GPU when the benefit outweighs the cost. Also consider that you get one SIMD unit per core, but typically only one GPU per chassis, and the GPU is at the moment a shared resource. This limits the number of jobs that you can run on a GPU. (The number of cores is increasing all the time, but the number of GPUs is not.) – Pseudonym Sep 3 '14 at 0:26
• Intel does not do much in terms of GPUs (apart from Larrabee/Knights Landing :) ), so I guess it is natural for them to try to push AVX instead. Although heavy AVX use may be very performant, it now results in downclocking on their newer CPUs, so they may be hitting limits with something. – nsandersen Jul 20 '17 at 11:38

Let's do a comparison. Intel's core i7-5960x has 8 cores, each with 4-wide (double precision) SIMD, running at 3 GHz (3.5GHz turbo), a 20M L3 cache, consumes 356mm^2 and 140W and costs \$1000. So 8*4*3*4 = 384 double precision GFlops. (The extra 4x is because you can do two fused-multiply-adds per vector lane per cycle.) It can do 768 single precision GFlops. That's about 1.08 DP GFlops/mm^2 and 2.75 DP GFlops/Watt. There's also about 57.5 KB/mm^2 of on-chip cache. NVidia's GeForce GTX Titan Black has 15 SMXs, each with 32-wide double precision SIMD, running at 890MHz (980MHz turbo), 3.5M of L1+L2 cache, consumes 561mm^2, 250W and costs \$1000. So 15*32*.89*4 = 1709 double precision GFlops. (Same 4x from two fused-multiply-adds per vector lane per cycle.) It can do 5126 single precision GFlops. That's about 3.05 DP GFlops/mm^2 and 6.8 DP GFlops/Watt. So 3x the DP floating point density per unit area and 3x the DP floating point power efficiency. And the tradeoff? 6.4 KB/mm^2 of on-chip cache. About 9x less dense than the CPU.