Automatic parallelization isn't a new thing by any means, and lots of questions have been asked about whether Java or C or C++ or a host of other languages or specific compilers (gcc 5 v 6, clang, Hotspot, etc.) support automatic parallelization. In many cases, the problem seems to be that compilers sometimes try to automatically parallelize code, but it's just a really hard problem due to memory guarantees (in a managed language) or pointer tracking (in an unmanaged one). The impression I get is that it's just tough to do in a language that was designed primarily for serial execution.

Let's say we were trying to build a programming language from the ground up specifically to simplify the task of automatic parallelization, vectorization, and distribution--from small loops/computations (SIMD CPU instructions, GPU kernels) up to large portions of code (threads, remote processes, cloud computing), we want to be able to automatically determine how a program could be broken up to be run in as parallel a manner as possible.

Some potentially relevant considerations might be:

  1. Would it be easier with managed or unmanaged memory? (in particular, thinking of how Rust does unmanaged memory but has built-in thread safety)

  2. Would it be easier with a purely (or mostly) functional language, or OOP, or imperative?

  3. How to incorporate non-determinism, such as user input, network traffic, or PRNG's?

What characteristics would it be important for such a language to have to make this task as simple and robust as possible?


closed as too broad by David Richerby, fade2black, Evil, Luke Mathieson, Andrej Bauer Nov 5 '17 at 13:39

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ Pretty much any purely functional language is trivially automatically parallelizable. The issue is granularity. Forking even extremely lightweight threads (e.g. GHC's sparks) is not worth it to execute a few machine instructions. You should look at GHC Haskell as its early history was in research on automatic parallelization, and its modern incarnation has, via the language or libraries, interesting approaches to every topic you mention. It's, of course, by no means exhaustive, so you should look elsewhere as well. $\endgroup$ – Derek Elkins Oct 25 '17 at 1:07