A talk from the OSS & Tooling 2 session at RSECon23 at Swansea University, on 2023-09-07.
Programme link: virtual.oxford...
Slides: drive.google.c...
Abstract:
Benchmarking has always been a crucial activity for HPC. Being able to systematically assess the performance of important algorithms on different HPC machines is needed for planning the next steps in algorithm development, evaluating available and upcoming hardware technologies, and procuring new HPC systems with confidence. The benchmarking process though has been somewhat of a dark art, usually involving convoluted, machine-dependent scripts and configurations that only the few can run, re-run, and reproduce their results. The situation is only becoming worse with the advent of the quest for exascale, with the available hardware technologies becoming more heterogeneous.
We will present how our project, which is part of the ExCALIBUR program, is set to address the problem of automation and reproducibility in benchmarking by creating an Open Source, user-friendly, automated framework based on Spack and ReFrame, for building and running benchmarks and collecting and visualising their results. This framework includes a still-growing suite of benchmark codes, as well as configurations to run them on popular HPC systems. It will also facilitate performance portability studies of codes across those different platforms.
As part of this project, we also aim to create a community of practice for benchmarking, which spans across ExCALIBUR, DiRAC, and other HPC developers and providers in the UK. We will show how we are making benchmarking straightforward for all by advertising our work in various fora, mailing lists and events, running training workshops, and inviting and collaborating with application and benchmark developers to add their benchmarks to our suite.
6 сен 2024
