MPI (Message Passing Interface) is the de facto standard for distributed-memory parallel programming and the backbone of virtually all large-scale HPC applications. This course introduces the fundamental concepts of distributed-memory computing and the full breadth of the MPI standard – from point-to-point and collective communication to derived datatypes, communicators, and one-sided communication – giving participants a solid foundation for writing scalable parallel programs on HPC clusters.
Both correct usage and performance considerations of MPI are covered; tracing tools for basic program analysis are also introduced.
It pairs naturally with the Introduction to Parallel Programming with OpenMP course and combines well with the Hybrid Programming in HPC – MPI+X course for participants wishing to combine both programming models.
Level: Beginner
Language: English
Price and Eligibility: Refer to the registration page for each event (generally free of charge for members of academia from Europe).
Knowledge
- Programming experience in C, C++, or Fortran (the majority of examples will be demonstrated in C/C++)
- Familiarity with the Linux command line, including remote login, file editing, and compilation
Technical
- A modern web browser (exercises run on NHR@FAU’s HPC clusters via JupyterHub – no local installation required)
After completing this course, you will be able to:
- Explain distributed-memory architecture and the role of MPI in HPC application development
- Write correct MPI programs using blocking and non-blocking point-to-point communication
- Apply collective communication operations for common parallel patterns such as broadcasts, reductions, and scatters
- Construct derived datatypes to communicate non-contiguous data structures efficiently
- Partition communicators and manage process topologies using sub-communicators
- Identify and address common MPI performance bottlenecks
- Use tracing tools to analyze and understand the runtime behavior of MPI programs
- Distributed-memory architecture and the MPI programming model
- Blocking and non-blocking point-to-point communication
- Collective communication: broadcasts, reductions, scatters, and gathers
- Derived datatypes for non-contiguous and heterogeneous data
- Communicators, process groups, and virtual topologies
- Performance considerations and communication optimization
- Program analysis with MPI tracing tools
- 2026, May 7-8: two-day online course
- 2025, Apr 9: full-day online course
- 2024, Apr 11: full-day online course
For an overview of all NHR@FAU courses, visit the course overview page.