What is the relation and difference between a programming model and a programming paradigm? (especially when talking about the programming model and the programming paradigm for a programming language.)
Wikipedia tries to answer my question in 1:
Programming paradigms can also be compared with programming models that are abstractions of computer systems. For example, the "von Neumann model" is a programming model used in traditional sequential computers. For parallel computing, there are many possible models typically reflecting different ways processors can be interconnected. The most common are based on shared memory, distributed memory with message passing, or a hybrid of the two.
But I don't understand it:
Is it incorrect that the quote in Wikipedia says "the 'von Neumann model' is a programming model", because I understand that the Von Neumann model is an architectural model from https://en.wikipedia.org/wiki/Von_Neumann_architecture?
Are the parallel programming models "typically reflecting different ways processors can be interconnected"? Or are parallel architectural models "reflecting different ways processors can be interconnected" instead?
In order to answer the question in 1, could you clarify what a programming model is?
Is it correct that a programming model provided/implemented by a programming language or API library, and such implementation isn't unique?
From Rauber's Parallel Programming book, "programming model" is an abstraction above "model of computation (i.e. computational model)" which is in turn above "architectural model". I guess that a programming model isn't just used in parallel computing, but for a programming language, or API library.
A programming model is implied by the system architecture. If your system architecture is a register machine, your programming model will consist of machine code operations on registers. If your architecture is a stack machine, your programming model will consist of stack operations. A Von Neumann architecture and a Harvard architecture wil have other programming models. Self modifying code p.e. wil be possible in a Von Neumann architecture but not in a Harvard architecture.
A programming paradigm is more highlevel: it is the way a problem is modelled (imperative or declarative, Object oriented, functional, logic,...). A single paradigm language supports one of these. Multiparadigm languages are more a sort of Swiss armyknive which take elements out of more paradigms.
Every architecture (and corresponding model) will have his own set of machine code instructions. This machine code language itself will follow the imperative paradigm (do this , do that, read register A, add the value to register B,... or put a value on top af the stack, put another value on top of the stack, add the two values on top..., etc...)
(At least I never saw a non-imperative hardware processor)
A high level language (of whatever paradigm) wil be compiled or interpreted to this machine code.
About parallelism: If we consider interconnected processors it wil be clear that the way they interconnect wil be part of the programming model. An old INMOS transputer p.e. connects with four other transputers. The machine code wil have instructions to communicate with the naburing transputers.
But also on recent systems the way to provide mutual exclusion will have to be resolved on low level. On a one processor system we will have to put the interrupts on and off when leaving or entering the critical section. On a multiprocessor system we will need a monoatomic 'test and set' instruction. This is part of the programming model.
Parallel computing paradigms are high level models to use parallelism. Think on languages who have threaded objects, or use semaphores and monitors as language elements.
When we program on different operating systems, different API's will be used. (or even if we program on the same system but we use an other library - a graphics library p.e.). This will change our programming model. the low level code will be different, but if there is a good abstraction (sort of code once, compile anywhere) this will be invisible in the hilevel language. If not, you will have to make small changes in your code. But since you will use the same hi level language, there will be no change of paradigm.
There is no exact answer to your question. The terms "programming model" and "programming paradigm" are not exact technical terms that have fixed definitions. Depending on a context, some authors might define "programming model" in some specific way, but that will usually turn out to cover only some aspects of what people understand under "programming model".
As a good rule of thumb, you should run away from anyone using the word "paradigm". (I was a graduate student when the word was severly overused.)
Nevertheless, it is still very useful to use these phrases, even though they are not very precise. They are helpful in explaining and organizing various aspects of computation. But keep in mind that there are no accepted mathematical definitions that entirely cover the usage. Therefore, if you ask very precise questions, you will not get very precise answers, but rather opinions and helpful explanations.
In particular, you asked: "What is the relation and difference between a programming model and a programming paradigm?" Well, that depends on who you ask and what decade you live in. Here is what my dictionary says:
paradigm – a worldview underlying the theories and methodology of a particular scientific subject
model – a simplified description, especially a mathematical one, of a system or process, to assist calculations and predictions
So, a paradigm is a broader concept than a model. In any case, I advise you to not think about your question as one of science because it is not.
Programming models bridge the gap between the underlying
hardware architectureand the supporting layers of software available to applications.
Programming models are different from both
programming languages and
application programming interfaces (
APIs). Specifically, a programming model is an abstraction of the underlying computer system that allows for the expression of both
In comparison, languages and APIs provide implementations of these abstractions and allow the algorithms and data structures to be put into practice – a programming model exists independently of the choice of both the programming language and the supporting APIs.
Programming models are typically focused on achieving increased developer productivity, performance, and portability to other system designs. The rapidly changing nature of processor architectures and the complexity of designing an exascale platform provide significant challenges for these goals. Several other factors are likely to impact the design of future programming models.
Source https://asc.llnl.gov/content/assets/docs/exascale-pmWG.pdf, "Advanced Simulation and Computing" (nuclear weapons)
More on twitter :