• Group
• Research
• Projects
• Software
• Teaching
• Talks & Guests
• Open Positions
• How to find us

Contact us

TU Wien
Faculty of Informatics
Institute of Computer Engineering
Research Group Parallel Computing

Favoritenstrasse 16
1040 Vienna
Austria

Phone: +43 1 58801 191 411
E-mail:

• Home
• Teaching
• Introduction to Parallel Computing (184.710)

Plan

Lecture with practical project part (VU), attendendance is mandatory and really recommended. Lectures by Jesper Larsson Träff.
Lectures on Mondays 15.15 - 17.00 in lecture hall "Informatikhörsaal". Additional slots for exercises Tuesdays, 15:15 - 17:00 (Informatikhörsaal). See TISS and TUWEL.

• Rules, regulations and important dates (slides from the "Vorbesprechung", Monday 4.3.2019)
• Current announcements.
• Course in TUWEL

Overview

The aim of this lecture is to given an introduction to parallel computing on different types of dedicated, parallel computing systems. This includes basic concepts on how to achieve and estimate parallel performance, and practical exposure to some of the most common, current programming models, frameworks and paradigms.
Contents: basic concepts, history, motivation. Shared-memory parallel computing, concrete languages: pthreads, OpenMP and Cilk. Distributed memory computing, concrete interface: MPI. New architectures (GPU and others), new languages, other paradigms.

Lectures / Exercise discussions (in lecture hall)

1. Monday, 04.03.2019: Planning, Intro, Basics
2. Monday, 11.03.2019: Principles (I): time, work, cost, speed-up, Amdahls Law, scalability
3. Monday, 18.03.2019: Principles (II): Patterns, tasks
4. Monday, 25.03.2019: Examples and Algorithms
5. Monday, 01.04.2019: Shared-memory systems, (p)threads
6. Monday, 08.04.2019: OpenMP
7. Tuesday, 09.04.2019: Feedback (lecture) on Exercise 1
8. Monday, 15.04.2019: No lecture (Easter holidays)
9. Monday, 22.04.2019: No lecture (Easter holidays)
10. Monday, 29.04.2019: OpenMP
11. Monday, 06.05.2019: Distributed memory systems
12. Monday, 13.05.2019: MPI
13. Tuesday, 14.05.2019: Feedback (lecture) on Exercise 2
14. Monday, 20.05.2019: MPI
15. Monday, 27.05.2019: MPI
16. Monday, 03.06.2019: Cilk/OpenMP tasks
17. Tuesday, 04.06.2019: Feedback (lecture) on Exercise 3
18. Monday, 10.06.2019: No lecture (Whitsun holiday)
19. Monday, 17.06.2019: More efficient shared-memory programming
20. Monday, 24.06.2019: Exam

Exercises / Projects (Hand-in/Abgabe via TUWEL)

1. Tuesday, 26.03.2019: Hand-in, Exercise 0, upload ssh key (Getting access to the system)
2. Tuesday, 02.04.2019: Hand-in, Exercise 1
3. Tuesday, 07.05.2019: Hand-in, Exercise 2
4. Tuesday, 14.05.2019: Hand-in, Project 1 (OpenMP)
5. Tuesday, 28.05.2019: Hand-in, Exercise 3
6. Tuesday, 04.06.2019: Hand-in, Project 2 (MPI)

Final exam

Monday 24.6.

Resources

• Asymptotic definitions; sums; recurrences
• Concepts and keywords for the lecture
• OpenMP standard and documents
• MPI standard and documents
• Merging in parallel, method 1
• Merging in parallel, method 2

Standards

Canonical, indispensable when programming and debugging

• OpenMP 4.5 Specification(was: OpenMP 4.0 Specification, OpenMP 3.1 Specification)
• MPI 3.1 Standard (was: MPI 3.0 Standard, MPI 2.2 Standard)
• CilkPlus resources (obsolete: Cilk 5.4.6 Reference Manual)

Programs

• mpihello.c
  Compile with mpicc -o mpihello -O3 mpihello.c, execute with mpirun -np 4 ./mpihello
• openmphello.c
  Compile with gcc -Wall -fopenmp -o openmphello -O3 openmphello.c, execute with ./openmphello -t 10
• pthreadshello.c
  Compile with gcc -Wall -o pthreadshello -O3 Opthreadshello.c -pthread, execute with ./pthreadshello -t 25
• cilkhello.c
  Compile with gcc -Wall -fcilkplus -O3 cilkhello.c -o cilkhello, execute with ./cilkhello
• cilkqsort.c
  Compile with gcc -Wall -fcilkplus -O3 cilkqsort.c -o cqs, execute with ./cqs -n 1000000 -s 2 (etc.)
• ompquick.c
  Compile with gcc -Wall -fopenmp -O3 ompqsort.c -o oqs, execute with ./oqs -n 1000000 -s 2 (etc.)

Literature

• Thomas Rauber, Gudula Rünger: Parallel Programming for Multicore and Cluster Systems. Second edition, Springer 2013.
  Good overview, complements lecture well, covers some topics/applications not in lecture (and the other way round).
• Bertil Schmidt, Jorge Gonzaalez-Domiinguez and Christian Hundt and Moritz Schlarb: Parallel Programming. Concepts and Practice. Morgan Kaufmann, 2018.
  Newer introduction, more focus on GPU programming (that will not be covered in the lecture).

Additional literature

• Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar: Introduction to Parallel Computing. Pearson Addison-Wesley 2003.
  Standard selection of material, good coverage. Beware, many detail mistakes/typos.

The lecture covers material not in these books (and vice versa). Reading at least one of these books is highly recommended.