Differences between programming model and programming paradigm?

Question

1. What is the relation and difference between a programming model and a programming paradigm? (especially when talking about the programming model and the programming paradigm for a programming language.)

2. Wikipedia tries to answer my question in 1:

   Programming paradigms can also be compared with programming models that are abstractions of computer systems. For example, the "von Neumann model" is a programming model used in traditional sequential computers. For parallel computing, there are many possible models typically reflecting different ways processors can be interconnected. The most common are based on shared memory, distributed memory with message passing, or a hybrid of the two.

   But I don't understand it:

   • Is it incorrect that the quote in Wikipedia says "the 'von Neumann model' is a programming model", because I understand that the Von Neumann model is an architectural model from https://en.wikipedia.org/wiki/Von_Neumann_architecture?

   • Are the parallel programming models "typically reflecting different ways processors can be interconnected"? Or are parallel architectural models "reflecting different ways processors can be interconnected" instead?

3. In order to answer the question in 1, could you clarify what a programming model is?

   Is it correct that a programming model provided/implemented by a programming language or API library, and such implementation isn't unique?

   From Rauber's Parallel Programming book, "programming model" is an abstraction above "model of computation (i.e. computational model)" which is in turn above "architectural model". I guess that a programming model isn't just used in parallel computing, but for a programming language, or API library.

Answer 1

A programming model is implied by the system architecture. If your system architecture is a register machine, your programming model will consist of machine code operations on registers. If your architecture is a stack machine, your programming model will consist of stack operations. A Von Neumann architecture and a Harvard architecture will have other programming models. Self modifying code p.e. wil be possible in a Von Neumann architecture but not in a Harvard architecture.

A programming paradigm is more highlevel: it is the way a problem is modelled (imperative or declarative, Object oriented, functional, logic,...). A single paradigm language supports one of these. Multiparadigm languages are more a sort of Swiss armyknive which take elements out of more paradigms.

Every architecture (and corresponding model) will have his own set of machine code instructions. This machine code language itself will follow the imperative paradigm (do this , do that, read register A, add the value to register B,... or put a value on top af the stack, put another value on top of the stack, add the two values on top..., etc...)

(At least I never saw a non-imperative hardware processor)

A high level language (of whatever paradigm) will be compiled or interpreted to this machine code.

About parallelism: If we consider interconnected processors it will be clear that the way they interconnect will be part of the programming model. An old INMOS transputer p.e. connects with four other transputers. The machine code wil have instructions to communicate with the naburing transputers.

But also on recent systems the way to provide mutual exclusion will have to be resolved on low level. On a one processor system we will have to put the interrupts on and off when leaving or entering the critical section. On a multiprocessor system we will need a monoatomic 'test and set' instruction. This is part of the programming model.

Parallel computing paradigms are high level models to use parallelism. Think on languages who have threaded objects, or use semaphores and monitors as language elements.

When we program on different operating systems, different API's will be used. (or even if we program on the same system but we use an other library - a graphics library p.e.). This will change our programming model. the low level code will be different, but if there is a good abstraction (sort of code once, compile anywhere) this will be invisible in the high level language. If not, you will have to make small changes in your code. But since you will use the same high level language, there will be no change of paradigm.

Answer 2

There is no exact answer to your question. The terms "programming model" and "programming paradigm" are not exact technical terms that have fixed definitions. Depending on a context, some authors might define "programming model" in some specific way, but that will usually turn out to cover only some aspects of what people understand under "programming model".

As a good rule of thumb, you should run away from anyone using the word "paradigm". (I was a graduate student when the word was severly overused.)

Nevertheless, it is still very useful to use these phrases, even though they are not very precise. They are helpful in explaining and organizing various aspects of computation. But keep in mind that there are no accepted mathematical definitions that entirely cover the usage. Therefore, if you ask very precise questions, you will not get very precise answers, but rather opinions and helpful explanations.

In particular, you asked: "What is the relation and difference between a programming model and a programming paradigm?" Well, that depends on who you ask and what decade you live in. Here is what my dictionary says:

paradigm – a worldview underlying the theories and methodology of a particular scientific subject

model – a simplified description, especially a mathematical one, of a system or process, to assist calculations and predictions

So, a paradigm is a broader concept than a model. In any case, I advise you to not think about your question as one of science because it is not.

Answer 3

Programming models bridge the gap between the underlying hardware architecture and the supporting layers of software available to applications.

Programming models are different from both programming languages and application programming interfaces (APIs). Specifically, a programming model is an abstraction of the underlying computer system that allows for the expression of both algorithms and data structures.

In comparison, languages and APIs provide implementations of these abstractions and allow the algorithms and data structures to be put into practice – a programming model exists independently of the choice of both the programming language and the supporting APIs.

Programming models are typically focused on achieving increased developer productivity, performance, and portability to other system designs. The rapidly changing nature of processor architectures and the complexity of designing an exascale platform provide significant challenges for these goals. Several other factors are likely to impact the design of future programming models.

Source https://asc.llnl.gov/content/assets/docs/exascale-pmWG.pdf, "Advanced Simulation and Computing" (nuclear weapons)

More on twitter : #ProgrammingModel

Programming paradigms

Imperative Programming with an explicit sequence of commands that update state.

Declarative Programming by specifying the result you want, not how to get it.

Structured Programming with clean, goto-free, nested control structures.

Procedural Imperative programming with procedure calls.

Functional (Applicative) Programming with function calls that avoid any global state.

Event-Driven Programming with emitters and listeners of asynchronous actions.

Flow-Driven Programming processes communicating with each other over predefined channels.

Logic (Rule-based) Programming by specifying a set of facts and rules. An engine infers the answers to questions.

Source https://cs.lmu.edu/~ray/notes/paradigms/

Thinking through these definitions helps give answers to the question.

One way to reason would be to say that programming models are closer to the design of a system / application and can be made of one or more programming paradigms which are close to a programming language.

This can be seen very clearly in data science applications where a number of technologies are used, each with their own programming paradigm, but overall the system the architect should keep a cohesive programming model.

More on this way of reasoning : https://twitter.com/semanticbeeng/status/1103027054302425089?s=20