Books about data structures and algorithms but specifically focusing on parallel gpgpu programming

Question

I'm looking for a book on the subject of algorithms and data structures, but I want to specifically learn about the parallel implementations. Books that teach CUDA and OpenCL usually have small sections that briefly touch on this subject, but they are mostly focused on technicalities. I'm looking for something more fundamental and theoretical.

I hope to later apply those techniques to problems in scientific computing, computational physics and simulations. If the book touches on those subjects, that's even better, but not strictly necessary.

Answer 1

I think you need to look at the HPC (High-Performance Computing) - because scientific computing requires so vast amount of resources now, that it'll be impossible to stay on the edge without access to really large machines.

However I don't think you'll find a comprehensive and refined description of algorithms and data structures, used in the HPC. It'll be easy to find some books about HPC, using this or similar keywords. Two books look promising to me - this introductory book and this book, mostly focused on GPGPU.

If you want to read something more theoretical, then probably it'll make sense to read scientific papers, for example ones from this collection. You'll easily find many HPC conferences as well, which usually happen every year, for example the SC'21 (November of this year).

Answer 2

For GPGPU focussed parallel programming I recommend

David B Kirk; Wen-mei W Hwu: Programming Massively Parallel Processors: A Hands-on Approach, 3/e, Morgan Kaufmann, 2016.

The examples are all CUDA based, but the algorithmic/data-structure ideas are all aimed at a much broader audience.

Disclaimer: Wen-mei Hwu and I were colleagues at University of Illinois, so I'm biased.