NHR@FAU Liaison Scientists

NHR@FAU PI: Prof. Dr. Petra Imhof, Professorship for Computational Chemistry, Computer-Chemistry-Center, and NHR@FAU

NHR activities, projects, and support

Frank Beierlein has benchmarked and tested Amber20 on the new RTX 3080 (TinyGPU) and Tesla A40 (Alex) GPUs, in addition to local GPUs (RTX A5000). As the new clusters use Slurm instead of torque as batch system, the submit scripts for different types of Amber jobs were adapted for Slurm. Frank watches the Amber list and the Amber literature to ensure being up-to-date with what happens in the Amber community.

Frank optimized the setup of Amber20 free energy thermodynamic integration simulations for efficient use of the new GPU cluster Alex (and the new nodes on TinyGPU) in tandem with Thomas Zeiser from the Systems & Services group of NHR@FAU. In his current research on DNA repair mechanisms, Frank develops high-quality parameters for damaged DNA bases which are part of a DNA strand that binds to a DNA repair enzyme. He adapted and refined the setup, simulation and analysis process for simulations in the field of alchemistic free energy methods, like thermodynamic integration, so that these simulations can now be run routinely on the NHR@FAU HPC clusters employing NVIDIA’s MPS server. Moreover, Frank optimized the the process of analyzing DNA/protein MD simulations and free energy simulation. He takes care that documentation, scripts, and inputs are as user-friendly as possible ensuring that even inexperienced users can apply advanced simulation techniques on HPC systems.

Teaching and training

Frank is closely involved in teaching computational chemistry courses/hands on user training for the FAU students at the Computer-Chemistry-Center (CCC): Moderne Softwareapplikationen, Molecular Modeling, Computational Chemistry, Bio-Organic & Bio-Inorganic LAB and Drug Discovery LAB.

NHR@FAU PI: Prof. Dr. Ulrich Rüde, Chair of Computer Science 10 (System Simulation), and NHR@FAU

NHR activities, projects, and support

Sebastian Kuckuk has already provided support in analyzing libxc that was focused on the overall workflow, the compute kernel composition, the maple-to-C scripts, and revealing potential for optimization. Based on this, he provided suggestions and a proof-of-concept implementation for employing code generation techniques using Python and SymPy.

Moreover, he provides consultation on the efficient use of GPUs. Noteworthy topics include porting legacy code to accelerators, profiling and performance analysis for GPU-enabled applications, employing hybrid (CPU-GPU) parallelization for application codes, and porting applications to energy-efficient SoC architectures. Sebastian has already worked with an MPI/OpenMP hybrid C++ code accelerated with CUDA and a C/C++/FORTRAN code that was later accelerated with OpenCL.

Sebastian is certified as NVIDIA Deep Learning Institute (DLI) University Ambassador.

Teaching and training

Sebastian completed multiple courses offered by the NVIDIA DLI on GPU programming and CUDA-accelerated applications that scale across multiple GPUs. He attained certification as DLI ambassador and was certified to teach the courses Fundamentals of Accelerated Computing with CUDA C/C++, Fundamentals of Accelerated Computing with CUDA Python, Accelerating CUDA C++ Applications with Multiple GPUs, and Scaling CUDA C++ Applications to Multiple Nodes.

Further, he developed a training unit on performance analysis for stencil codes on GPUs; this entailed implementing benchmark cases, profiling different configurations, data editing, and creating slides. A first version was presented as part of the Programming Techniques for Supercomputers (PTfS) lecture at FAU. In addition, he conceptualized a new workshop on performance portability and programmer productivity and realized first teaching units as interactive Jupyter notebooks.

NHR@FAU PI: Prof. Dr. Heinrich Sticht, Professorship for Bioinformatics, and NHR@FAU

NHR activities, projects, and support

Anselm Horn was granted an early-user-access for the new clusters Alex (GPU) and Fritz (CPU) in order to test the large number of A40 cards in simulations. The project aimed at investigating the feasibility of modelling the binding of a ligand to a single chain antibody fragment (scFv); for that, many different initial ligand orientations in the simulation setup ensured truly independent simulation runs. Since this project was the first larger-scale application of Amber on Alex and Fritz, the experience gained is most valuable for HPC users and NHR@FAU projects.

Using his expertise on Amber force field parameterization, Anselm developed 14SB-compatible parameters for a PEG3 and a PEG6 spacer unit. In conjunction with earlier developed parameters for isopeptide bonds, he was able to investigate the dynamics of a covalently coupled PEG-containing peptide linker around insulin-like growth factor using the A100 GPU cards of TinyGPU.

Anselm is actively involved in the Amber Mailing List helping Amber users all over the world to overcome problems, especially in the area of parameter generation. In addition, he is interested in Research Data Management for data from Molecular Dynamics simulations.

Teaching and training

In order to provide a first help for new users, Anselm works on an extension of the Tips & Ticks section for Amber with focus on practical aspects with regard to the new compute clusters Alex and Fritz.

NHR@FAU PI: Prof. Dr. Rainer Böckmann, Professorship for Computational Biology, and NHR@FAU

NHR activities, projects, and support

Marius Trollmann took an active part in the test phase of the recently established GPU cluster Alex at NHR@FAU. He provided a wide-range set of systems to analyze the performance of the simulation software GROMACS on the available A40 and A100 GPUs. These results help Gromacs users at NHR@FAU to get a reliable estimation of their required computation time, to achieve reliable results at a reasonable time scale, and will further help potential users to exploit their granted resources more efficiently.

One of his projects approaches the toxicity of hundreds of antimicrobial peptides (AMPs) against red blood cells in a high-throughput method. The huge number of simulation systems requires the extensive optimized use of multiple GPUs. Another project considers the study of mRNA-based vaccines: Approximately 28,500 GPU-hours were spent to characterize the structure and the physico-chemical properties of the lipid nanoparticles (LNPs) used in the BioN-Tech&Pfizer vaccine employing classical molecular dynamics (MD) simulations. The computationally demanding atomistic simulations of the LNPs with a size between three and seven Mio atoms requires extensive use of multiple A40 and A100 GPUs.

Teaching and training

Marius is involved in the master module Orientierungsmodul Strukturbiologie I and Orientierungsmodul Strukturbiologie II where participants get in touch with a realistic research hypothesis for which they then prepare and perform own simulations. He also contributes to the practical part of the module Molecular Dynamics Simulation of Biological Membranes which deals in particular with the programming of a simple MD-Engine employing Python and the application of free-energy calculations with Gromacs.

NHR@FAU PI: Prof. Dr. Ulrich Rüde, Chair of Computer Science 10 (System Simulation), and NHR@FAU

NHR activities, projects, and support

Rafael Ravedutti Lucio Machado has already provided support in analyzing libxc that was focused on the overall workflow, the compute kernel composition, the maple-to-c scripts and revealing potential for optimization. Based on this, he provided suggestions and a proof-of-concept implementation by extracting the parts of interest from libxc and providing an easy way to replace and evaluate the correctness of these kernels with the new implementation using Python and SymPy.

Currently, he is working on the development of MD-Bench, a proxy-application to evaluate the performance of Molecular Dynamics kernels with several strategies and configurations on different machines. In 2021, the implementation of the GROMACS MxN pair list scheme was introduced into MD-Bench, making it the first proxy app with such optimization. This is particularly important to study the performance improvements obtained with it and evaluate whether this optimization can be leveraged into other particle simulation scenarios such as Discrete Element Methods (DEM), Smoothed-Particle Hydrodynamics (SPH) and other Molecular Dynamics simulations beyond GROMACS (such as the material modeling simulation cases that currently can only be executed in LAMMPS, for instance).

In addition, he is developing P4IRS, a framework for the code generation of particle simulation kernels targeting multiple hardware such as CPUs and GPUs. The framework uses a simple symbolic description in Python to setup the simulation and to describe the potentials and/or force fields, and then generates optimal code for the chosen hardware using domain-knowledge from such simulations achieved in MD-Bench experimentation and performance studies.

Teaching and training

Rafael is responsible for the Exercises for High-end Simulation and Practice (HESP) lecture from the Chair of Computer Science 10 (System Simulation). The exercises consist of implementing particle simulations (MD, DEM, and/or SPH) to execute efficiently on GPUs with domain-specific optimizations such as the Linked Cells and Verlet Lists. He also supervises the final projects for the lecture which can focus on implementing DEM or SPH, as well as porting the application to execute in multiple GPUs with MPI. Rafael also supervised a project which consisted of porting the Lennard-Jones kernel in MD-Bench to CUDA in order to execute it efficiently on NVIDIA GPUs.

NHR@FAU PI: Prof. Dr. Andreas Görling, Chair of Theoretical Chemistry, and NHR@FAU

NHR Activities, projects, and support

Egor Trushin performed benchmark tests of GPU-accelerated Vienna Ab initio Simulation Package (VASP) on TinyGPU and Alex GPGPU clusters. Performance for various computational setups was analysed including various GPU models and number of used GPUs. Comparison with the CPU-executed version was also done. Moreover, he participated in the development and implementation of new quantum chemistry and density functional methods in the Molpro quantum chemistry package. Egor’s main areas of expertise are electronic structure calculations for molecules and solids, code development with Python and FORTRAN, and parallel programming using openMP und MPI.

NHR@FAU PI: Prof. Dr. Erik Bitzek, Chair of General Materials Properties / Prof. Dr. Michael Zaiser, Chair for Materials Simulation, and NHR@FAU

NHR activities, projects, and support

Samaneh Nasiri joined NHR@FAU as a Liaison scientist (Department of Materials Science and Engineering) on October 01, 2021. She contributed to preparing a DFG-Funded proposal entitled Molecular simulation of deformation processes in composites of light metals and covalent nanoparticles. This project aimed to find out how embedded nanoparticles change the deformation and fracture properties of the resulting nanocomposites and identify optimal parameters for mechanical property enhancement. This study involved large-scale atomistic simulation, which was planned to be carried out using the computation resources at NHR@FAU. The research was conducted in collaboration with colleagues in the UK and China who investigated the same metal-carbon systems experimentally and with other simulation methods such as density functional theory. Samaneh Nasiri has conducted preliminary investigations and proof-of-concept simulations required for the project proposal. In 2022, Samaneh left NHR@FAU.