NHR PerfLab Seminar: Mixing precisions in numerical algorithms – HPC perspective with modern hardware context (July 16, online)
Title: Mixing precisions in numerical algorithms: HPC perspective with modern hardware context
Speaker: Piotr Luszczek, MIT Lincoln Lab, The University of Tennessee Knoxville
Date and time: Tuesday, July 16, 2024, 2:00-3:00 p.m. CEST
Location: Online via Zoom: https://fau.zoom.us/j/68603607738
Abstract
IEEE 754 standard for floating-point arithmetic has served the scientific computing community for decades. The advent of deep learning marked the race to the bottom for the forever smaller bit representations of neural network weights and gradients. We see the ever more limited data formats could still be successfully used in training of neural networks of increasing size and complexity. Even smaller bit counts are applied for inference tasks that, unlike training, come at the speed of internet search queries. Thus the landscape of floating-point formats expanded initially just to be subsequently fractured partially due to the reinvigorated scene of the hardware startups, which in turn applied pressure on the numerical community to develop new mixed precision algorithms and their analysis methods.In this talk, I will expand on these recent shifts in hardware’s floating-point implementations and then focus on my contributions to the algorithmic side of this burgeoning field. More specifically, I will cover a range of numerical methods, their convergence properties, and performance considerations when applied to HPC benchmarking.
Short Bio
Piotr Luszczek received the BS and MSc degrees in computer science from the AGH University of Science and Technology in Kraków, Poland, and the Ph.D. degree in computer science from the University of Tennessee Knoxville. He is currently a Research Assistant Professor and Adjunct Associate Professor with Innovative Computing Laboratory, University of Tennessee, Knoxville’s Tickle College of Engineering. His research interests include benchmarking, numerical linear algebra for high-performance computing, automated performance tuning for modern hardware, and stochastic models for performance.
For a list of past and upcoming NHR PerfLab seminar events, see: https://hpc.fau.de/research/nhr-perflab-seminar-series/