Parallel Computing 101: An overview of multiprocessor, multicore, and GPU programming

Speaker:  Javier Gonzalez-Sanchez – Tempe, AZ, United States
Topic(s):  Software Engineering and Programming


Traditionally, software has been written for serial computing where a computational problem is broken into a discrete series of instructions; then these instructions are executed sequentially one after another on a single processor. Parallel computing is the simultaneous use of multiple compute resources to solve a computational problem. In parallel computing a problem is broken into discrete parts that can be solved concurrently; then each part is further broken down to a series of instructions, and instructions from each part execute simultaneously on a different compute resource. Parallel computing allows for the shortening of computing time with potential cost savings (for instance, for image rendering or video processing), and allows for the solving of large and complex problems requiring petaflops of computing speed (for instance, in search engines or databases processing millions of transactions every second).

This lecture introduces the principles of parallel computing beginning with an introduction to what parallel computing is, how it is used, and concepts and terminology associated. Then variations in system architectures with multiple compute resources are presented, including multiple functional units (such as GPUs), multiple execution units (such as multiple cores), multiple processors, and networks connecting multiple stand-alone computers configured as a computer cluster. The following programming models are explored:

* Threads model using POSIX Threads and OpenMP

* Message Passing using MPI

* Single-Instruction Multiple-Data (SIMD) using GPU programming. 

A series of practical examples of how to parallelize representative examples of code are presented for each model. The lecture concludes with a discussion of a number of challenges related to designing and running parallel programs. 

About this Lecture

Number of Slides:  50
Duration:  60 minutes
Languages Available:  English, Spanish
Last Updated: 

Request this Lecture

To request this particular lecture, please complete this online form.

Request a Tour

To request a tour with this speaker, please complete this online form.

All requests will be sent to ACM headquarters for review.