Accelerated C++ Programming with CUDA Training Course

Introduction to GPU-Accelerated Computing

• Heterogeneous computing concepts and CPU-GPU architecture
• CUDA execution flows and memory spaces
• Compiling CUDA C++ code with nvcc and CMake
• Validating the GPU development environment

Parallel Algorithms with Thrust and CUB

• GPU-accelerated sorting, reduction, and transformation
• Adapting STL algorithms for GPU execution
• Thrust device vectors and execution policies
• CUB device-wide primitives for custom pipelines

GPU Memory Architecture and Management

• Understanding global, constant, and texture memory types
• Managing explicit device memory allocation and transfers
• Utilizing Unified Memory for streamlined data access
• Optimizing memory coalescing and access patterns

Asynchronous Execution with CUDA Streams

• Creating and managing CUDA streams
• Overlapping kernel execution with data transfers
• Using CUDA events for dependency management
• Tuning stream priorities and concurrency

Writing Custom CUDA Kernels

• The SIMT programming model and warp execution
• Kernel launch configurations and grid-stride loops
• Thread indexing and multidimensional grids
• Error handling and CUDA runtime API checks

Thread Hierarchy and Execution Model

• Grids, blocks, and threads in device code
• Warp-level primitives and ballot operations
• Block-level synchronization and barriers
• Analyzing occupancy and resource utilization

Cooperative Groups for Flexible Parallelism

• The cooperative_groups API and various group types
• Thread-block tiles and tiled_partition techniques
• Grid-level cooperative launches
• Multi-grid synchronization patterns

Shared Memory Optimization Techniques

• Shared memory banks and avoiding bank conflicts
• Tiling strategies for matrix operations
• Using shared memory as a user-managed cache
• cuda::shared_memory_mdspan for multidimensional views

Kernel Fusion and Advanced Parallel Patterns

• Fusing kernels to minimize launch overhead
• Implementing scan, reduce-by-key, and segmented algorithms
• Utilizing atomic operations and lock-free data structures
• Warp-aggregated atomics for improved throughput

Profiling and Optimization with Nsight Systems

• Timeline analysis of CPU and GPU activity
• Identifying memory transfer bottlenecks
• Profiling kernel performance and occupancy
• Iterative optimization using Nsight Compute

Modern C++ Features in CUDA Device Code

• Applying lambdas, constexpr, and auto within kernels
• C++17 parallel algorithms and execution policies
• C++20 concepts and ranges on device
• C++23 support in nvcc and CCCL 3.x

CUDA Graphs and Advanced Asynchrony

• Defining and launching CUDA graphs
• Capturing graphs from stream execution
• Updating graphs and handling conditional execution nodes
• Reducing launch latency for iterative workloads

Integration Patterns for Existing Applications

• Encapsulating GPU code behind C++ interfaces
• Managing multi-GPU and NUMA systems
• Integrating build systems with CMake and CUDA
• Debugging device code with cuda-gdb

Summary and Best Practices

• Deciding between Thrust, CUB, and custom kernels
• Ensuring performance portability across GPU architectures
• Organizing code and applying RAII for CUDA resources
• Exploring next steps and advanced CUDA learning paths