Automatic parallelization isn't a new thing by any means, and lots of questions have been asked about whether Java or C or C++ or a host of other languages or specific compilers (gcc 5 v 6, clang, Hotspot, etc.) support automatic parallelization. In many cases, the problem seems to be that compilers sometimes try to automatically parallelize code, but it's just a really hard problem due to memory guarantees (in a managed language) or pointer tracking (in an unmanaged one). The impression I get is that it's just tough to do in a language that was designed primarily for serial execution.

Let's say we were trying to build a programming language from the ground up specifically to simplify the task of automatic parallelization, vectorization, and distribution--from small loops/computations (SIMD CPU instructions, GPU kernels) up to large portions of code (threads, remote processes, cloud computing), we want to be able to automatically determine how a program could be broken up to be run in as parallel a manner as possible.

Some potentially relevant considerations might be:

  1. Would it be easier with managed or unmanaged memory? (in particular, thinking of how Rust does unmanaged memory but has built-in thread safety)

  2. Would it be easier with a purely (or mostly) functional language, or OOP, or imperative?

  3. How to incorporate non-determinism, such as user input, network traffic, or PRNG's?

What characteristics would it be important for such a language to have to make this task as simple and robust as possible?

  • 1
    $\begingroup$ Pretty much any purely functional language is trivially automatically parallelizable. The issue is granularity. Forking even extremely lightweight threads (e.g. GHC's sparks) is not worth it to execute a few machine instructions. You should look at GHC Haskell as its early history was in research on automatic parallelization, and its modern incarnation has, via the language or libraries, interesting approaches to every topic you mention. It's, of course, by no means exhaustive, so you should look elsewhere as well. $\endgroup$ – Derek Elkins left SE Oct 25 '17 at 1:07