NHR PerfLab Seminar Talk: ALTO: Adaptive Linearized Storage of Sparse Tensors
We are happy to announce a talk by Jan Laukemann (Intel Labs) in the NHR PerfLab seminar.
Title: ALTO: Adaptive Linearized Storage of Sparse Tensors
Abstract:
The analysis of high-dimensional sparse data is becoming increasingly popular in many important domains. However, real-world sparse tensors are challenging to process due to their irregular shapes and data distributions. We propose the Adaptive Linearized Tensor Order (ALTO) format, a novel mode-agnostic (general) representation that keeps neighboring nonzero elements in the multi-dimensional space close to each other in memory. To generate the indexing metadata, ALTO uses an adaptive bit encoding scheme that trades off index computations for lower memory usage and more effective use of memory bandwidth. Moreover, by decoupling its sparse representation from the irregular spatial distribution of nonzero elements, ALTO eliminates the workload imbalance and greatly reduces the synchronization overhead of tensor computations. As a result, the parallel performance of ALTO-based tensor operations becomes a function of their inherent data reuse. On a gamut of tensor datasets, ALTO outperforms an oracle that selects the best state-of-the-art format for each dataset, when used in key tensor decomposition operations. Specifically, ALTO achieves a geometric mean speedup of 8X over the best mode-agnostic format, while delivering a geometric mean compression ratio of more than 4X relative to the best mode-specific format.
Date and time: Tuesday, June 1, 2 p.m. – 3 p.m.
Slides: 20210601_NHR_PerfLab_pdf
Paper preprint: https://arxiv.org/abs/2102.10245
Short Bio:
Jan Laukemann is a Research Scientist and PhD Intern at Intel Labs (Intel PCL) where he focuses on High Performance Computing (HPC), Performance Engineering, and Performance Analysis. He works on application optimization for HPC systems and novel algorithms for scalable linear algebra, tensor decomposition and graph computations. His research interests primarily include x86 and non-x86 computer architectures, their performance behavior on the node level, and vectorization techniques. He received his Master’s degree in computer science from the University of Erlangen-Nürnberg (FAU) and currently pursues his PhD with the Erlangen National High Performance Computing Center (NHR@FAU), Germany.