Please suggest some books on Parallel Algorithms which teach efficient programming techniques like the ones taught in Udacity course(Introduction to Parallel Programming with CUDA). Also wanted to know that from which reference book or papers are the concepts in the udacity course on Parallel Computing taught...? I wish to study more about such techniques which will make me an efficient parallel programmer...!

  • $\begingroup$ I just wonder why someone gave a negative vote to this question without giving any comment. The instructions for the tag "reference-request" specify "if you ask for pointers to material for self-study". Accordingly, this question looks perfectly legitimate. $\endgroup$
    – J.-E. Pin
    Commented Aug 23, 2013 at 11:11

2 Answers 2


Also wanted to know that from which reference book or papers are the concepts in the udacity course on Parallel Computing taught...?

The History of Parallel Computing goes back far in the past, where the current interest in GPU computing was not yet predictable. Some important concepts date back to that time, with lots of theoretical activity between 1980 and 1990. Compared to the complicated trade-offs for current heterogenous systems, those old theoretical algorithms and concepts feel nice, powerful and easy to understand. Uzi Vishkin is a researcher who worked during that period, who still believes in those concepts (without overly compilicated trade-offs):

Alas, the software spiral is now broken: (a) nobody is building hardware that provides improved performance on the old serial software base; (b) there is no broad parallel computing application software base for which hardware vendors are committed to improve performance; and (c) no agreed-upon architecture currently allows application programmers to build such software base for the future

I recently learned about those old concepts, and corresponding interesting survey articles:

Parallel Algorithms (64 pages)
by Guy E. Blelloch and Bruce M. Maggs – 1996

Thinking in Parallel: Some Basic Data-Parallel Algorithms and Techniques (104 pages)
by Uzi Vishkin – October 12, 2010

A Survey of Parallel Algorithms for Shared-Memory Machines (70 pages)
by Richard M. Karp and Vijaya Ramachandran – March 1988

Limits to Parallel Computation: P-Completeness theory (327 pages)
by R Greenlaw, H Hoover, W Ruzzo – 1995

Parallel Algorithms (347 pages)
by H Casanova, A Legrand, Y Robert – 2008


Programming Massively Parallel Processors is a good read. NVIDIA has also got some good tutorials. Amazon has got pretty good list.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.