AI Performance - rFronteddu/general_wiki GitHub Wiki

collective aggregation operations like all-reduce, all-to-all, and all-gather, which are extensively during model training and inference

• Ch.1-2 AI Systems Overview
• Ch.3 OS Docker, and Kubernetes Tuning for GPU‐Based Environments
• Ch4. Tuning Distributed Network Communication
• Ch5. GPU-Based Storage I/O Optimizations
• Ch.8 Occupancy Tuning, Warp-Efficiency, ILP