Is the way an OS schedule threads related to parallel computing?

Question

From Wikipedia:

Operating systems schedule threads in one of two ways:

Preemptive multitasking is generally considered the superior approach, as it allows the operating system to determine when a context switch should occur. The disadvantage of preemptive multithreading is that the system may make a context switch at an inappropriate time, causing lock convoy, priority inversion or other negative effects, which may be avoided by cooperative multithreading.

Cooperative multithreading, on the other hand, relies on the threads themselves to relinquish control once they are at a stopping point. This can create problems if a thread is waiting for a resource to become available.

I wonder if in parallel computing (writing and running parallelized programs in OpenMP, OpenMPI, pThread), which of Cooperative multithreading and Preemptive multitasking is/are used, or does the way OS scheduling threads have nothing to do with the multi-process or multi-thread within a parallelized program?

Answer 1

There is a relationship between what Wikipedia is calling "preemptive multitasking" and running multiple threads in parallel, but it's somewhat indirect.

Suppose you have two threads of computation that are independent (the memory locations they touch are non-overlapping). Then we say that those two threads can run concurrently. "Preemptive multitasking" is a way of giving the illusion that the threads are running concurrently, even though we only have a single physical processor. The single processor gets interrupted (usually by some timer hardware) every once in a while (e.g., 100 times a second), and chooses which thread to run. Since the processor is switching back and forth so quickly it is hard to tell that you aren't running simultaneously on two (slow) processors.

If you actually run the two threads on two different processors then you are really running in parallel, and there is no switching back and forth to share a single resource.

The similarity comes up because in both true parallelism and preemptive multitasking any ordering of the instructions between the two threads is possible. Thus if there is any sharing of resources, memory, or locks you need to reason about the correctness of that sharing by proving that nothing bad can happen in any possible interleaving.

With cooperative multithreading you break the illusion of having multiple processors. If a thread gets scheduled it runs as long as it wants to (and knows that the other thread is not running) and only gives up the processor by calling some system call (often called something like yield().) On the one hand, you've lost the illusion of two processors, which is problematic if one of the threads forgets to yield(). The advantage is that you know when the the threads are switching back and forth, so the number of possible interleavings you need to reason about is much smaller.