High performance distributed systems are used to execute scientific applications to enable discovery at very large scales. This class will explore the software and hardware principles needed to design high performance applications from bottom to top. We will explore low level programming of vector, multicore, and GPU systems, and then scale up to use large clusters through distributed programming models. A final project will tie it all together by evaluating and optimizing a scientific application at scale.

Instructors

	Prof. Douglas Thain (dthain@nd.edu) Drop-In Hours: 1:00-3:30 Tuesdays Office: 206B Cushing
	Grad TA: Jin Zhou (jzhou24@nd.edu) Drop-In Hours: 11:30-1:00 Wednesdays Office: 254 Cushing (Student Lounge)

Course Organization

• Syllabus
• General Instructions for Assignments
• Course Canvas Page
• #hpds-fa26 Slack Channel

Computing Resources

• Notre Dame Center for Research Computing (CRC)
• NSF ACCESS Infrastructure

Course Schedule

Week	Monday	Wednesday	Friday
Aug 24	Class Overview Slides	The Whole Stack Slides	SIMD Parallelism Reading: SIMD Slides: SIMD Examples: SIMD
Aug 31	SIMD Contd A0: Warmup Due	OpenMP Reading: OpenMP Basics Tutorial: OpenMP	OpenMP Examples: OpenMP
Sep 7	OpenMP Examples: OpenMP A1: SIMD Parallel Due	OpenMP Slides: Benchmarks Examples: Benchmarks	Short Demos 1
Sep 14	Performance Slides: Perf Eval 1	Performance Slides: Perf Eval 1	Performance Slides: Perf Eval 2 Docs: Linux Perf
Sep 21	GPU Architecture Slides: GPU Architecture Reading: CUDA A2: OpenMP Assignment Due	CUDA Programming Slides: CUDA 1 Examples: CUDA	CUDA Programming Slides: CUDA 2 Examples: CUDA
Sep 28	Guest Lecture: Prof Kogge (CSE) Slides: Exascale	Guest Lecture: Scott Hampton (CRC) Resources: OpenOnDemand OpenWebUI CRC Docs	Short Demos 2
Oct 5	Project Overview Slides: Misc and Project A3: CUDA Assignment Due	HTCondor Slides: HTCondor	HTCondor
Oct 12	Workflow Systems Slides: Makeflow Project Bids Due	Makeflow + TaskVine	Short Demos 3
Oct 19	Fall Break
Oct 26	TaskVine A4: HTCondor Assignment Due	Parsl	FaaS
Nov 2	Software Environments Project Checkpoint 1	Pip, Conda, and Friends	Containers
Nov 9	Storage Performance	Parallel Filesystem	Wide Area Data
Nov 16	Supercomputing Week Project Checkpoint 2		Short Demos 4
Nov 23	Beautiful Data	Thanksgiving Break
Nov 30	Project Presentations
Dec 7	Project Presentations	Project Presentations Final Project Due	Final Exam, 7:30PM

Readings and Reference

• SIMD - SIMD Parallelism - Chapter 10.1-10.6 of Algorithmica HPC by Sergey Slotin
• SIMD - GCC Vector Instructions
• OpenMP - The Art of HPC, Volume 2, Chapter 17-27 by Victor Eijkhout
• OpenMP - A "Hands-On" Introduction to OpenMP, Tim Mattson and Larry Meadows
• OpenMP - Various Presentations at OpenMP Foundation
• CUDA - NVIDIA CUDA Programming Guide
• CUDA - Programming Massively Parallel Processors: A Hands-On Approach, by Wen-mei Hwu, David B. Kirk, Izzat El Hajj, 2022. (at your favorite bookstore)
• HTCondor - HTCondor Quick Start
• HTCondor - HTCondor Users Manual
• HTCondor - HTCondor at Notre Dame
• Makeflow - Makeflow User's Manual
• TaskVine - TaskVine User's Manual
• Parsl - Parsl Parallel Scripting in Python
• Apptainer - Apptainer User Guide
• Data Visualization - and other works by Edward Tufte Graphics Press, 2006. The Visual Display of Quantitative Information