Plan
Lecture with practical project part (VU), this semester in distance-learning mode, with (streamed) recordings of the lecture every week. Lectures by Jesper Larsson Träff.Lectures on Mondays 15:00 - 17:00 in lecture hall "FAV 01". See TISS and TUWEL for additional information.
- Rules, regulations and important dates (slides from the "Vorbesprechung", Monday 1.3.2021)
- Current announcements.
- Course in TUWEL
Overview
The aim of this lecture is to given an introduction to parallel computing on different types of dedicated, parallel computing systems. This includes basic concepts on how to achieve and estimate parallel performance, and practical exposure to some of the most common, current programming models, frameworks and paradigms.Contents: Basic concepts, history, motivation. Shared-memory parallel computing, concrete languages: pthreads, OpenMP (and Cilk). Distributed memory computing, concrete interface: MPI. New languages, other paradigms.
Lectures / Exercise discussions (in lecture hall)
- Monday, 1.03.2021: Planning, Intro, Basics
- Monday, 8.03.2021: Principles (I): time, work, cost, speed-up, Amdahls Law, scalability
- Monday, 15.03.2021: Principles (II): Patterns, tasks
- Monday, 22.03.2021: Examples and Algorithms
Monday, 29.03.2021: No lecture (Easter holidays)Monday, 5.04.2021: No lecture (Easter holidays)- Monday, 12.04.2021: Shared-memory systems, (p)threads
- Monday, 19.04.2021: OpenMP
- Monday, 26.04.2021: OpenMP
- Monday, 03.05.2021: OpenMP
- Monday, 10.05.2021: Distributed memory systems
- Monday, 17.05.2021: MPI
Monday, 24.05.2021: No lecture (Whitsun holiday)- Monday, 31.05.2021: MPI
- Monday, 7.06.2021: MPI
- Monday, 14.06.2021: TBA
Exercises / Projects (Hand-in/Abgabe via TUWEL)
- Monday 15.3.: Exercise 0 (getting access to system, ssh key). Hand-in 22.3. (via TUWEL)
- Monday 12.4.: Exercise 1 (basic concepts). Hand-in 26.4. (firm deadline) (via TUWEL).
- Monday 3.5: Exercise 2 (shared-memory, OpenMP programming exercises). Hand-in 17.5. (via TUWEL).
- Monday 17.5.: Exercise 3 (distributed memory, MPI programming exercises). Hand-in 7.6. (via TUWEL).
Exams
Written exam (Klausur) when possible, see TISSResources
- Asymptotic definitions; sums; recurrences
- Concepts and keywords for the lecture
- OpenMP standard and documents
- MPI standard and documents
- Merging in parallel, method 1
- Merging in parallel, method 2
Standards
Canonical, indispensable when programming and debugging
- OpenMP 4.5 Specification(was: OpenMP 4.0 Specification, OpenMP 3.1 Specification)
- MPI 3.1 Standard (was: MPI 3.0 Standard, MPI 2.2 Standard)
- CilkPlus resources (obsolete: Cilk 5.4.6 Reference Manual)
Programs
mpihello.c
Compile withmpicc -o mpihello -O3 mpihello.c
, execute withmpirun -np 4 ./mpihello
openmphello.c
Compile withgcc -Wall -fopenmp -o openmphello -O3 openmphello.c
, execute with./openmphello -t 10
pthreadshello.c
Compile withgcc -Wall -o pthreadshello -O3 Opthreadshello.c -pthread
, execute with./pthreadshello -t 25
cilkhello.c
Compile withgcc -Wall -fcilkplus -O3 cilkhello.c -o cilkhello
, execute with./cilkhello
cilkqsort.c
Compile withgcc -Wall -fcilkplus -O3 cilkqsort.c -o cqs
, execute with./cqs -n 1000000 -s 2
(etc.)ompquick.c
Compile withgcc -Wall -fopenmp -O3 ompqsort.c -o oqs
, execute with./oqs -n 1000000 -s 2
(etc.)
Literature
- Thomas Rauber, Gudula Rünger: Parallel Programming for Multicore and Cluster Systems. Second edition, Springer 2013.
Good overview, complements lecture well, covers some topics/applications not in lecture (and the other way round). -
Bertil Schmidt, Jorge Gonzaalez-Domiinguez and
Christian Hundt and Moritz Schlarb:
Parallel Programming. Concepts and Practice. Morgan Kaufmann, 2018.
Newer introduction, more focus on GPU programming (that will not be covered in the lecture).
Additional literature
- Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar: Introduction to Parallel Computing. Pearson Addison-Wesley 2003.
Standard selection of material, good coverage. Beware, many detail mistakes/typos.