BLG 562E - Paral.Comp.for GPUs Using CUDA

Course Objectives

Understanding the CUDA programming model and GPU micro-architecture.
Studying and understanding the performance of CUDA programs and learning to develop efficient parallel programs using CUDA.

Course Description

History of GPUs. Principles of parallel algorithm design. CUDA programming model. GPU micro- architecture. Occupancy, GPU performance and performance analysis and debugging tools. Branch divergence and control flow optimizations. GPU memory system. Memory system optimizations. Unified Memory and CPU-GPU cooperative computing. Synchronization, atomics, cache coherence, memory consistency. Parallel patterns, Histogram, Prefix-sum, etc. Dynamic parallelism, streams, multi-GPU programming. CUDA libraries, CuBlas, CuDNN, NPP, NvGraph, cuFFT, NCCL, cuSPARSE, etc. Other GPU programming environments, OpenCL and HSA.