scieee AI-readable full text Open interactive document viewer

Reasonable Performance Computing SIG (SIG-RPC): Help Identify and Document Performance Traps in Research Software

FitzJohn, Rich; Heywood, Peter; Chisholm, Robert

Publisher
Zenodo
DOI
10.5281/zenodo.17967505
Repository
zenodo.org
Language
en

Abstract

Most research software is written by researchers who are not professionally trained programmers. This often leads to subtle but significant performance pitfalls—small coding choices, such as inefficient data structures, that can have an outsized impact on execution time. Even experienced developers struggle with performance issues, as language- and library-specific nuances are often learned through experience rather than formal documentation. Unreasonably slow research software harms the productivity of the researcher using it and wastes energy and shared resources (like HPC systems), which can harm the productivity of other researchers too.The Reasonable Performance Computing SIG (SIG-RPC) was founded to build a community around the research, development, and advocacy of software performance best practices. A key part of this effort is curating a knowledge base of common performance traps and profiling tools to help researchers and developers write faster, more efficient code.This workshop will introduce SIG-RPC and its knowledge base, guiding attendees through the process of contributing performance trap nominations and mini-guides based on their own expertise. Participants will engage in hands-on exercises such as:Reviewing existing guides – Are they accessible to a broad audience, including non-experts?Submitting new performance traps – Are there common pitfalls in your favourite language or library that haven’t yet been nominated or documented?Contributing new guides – Can you document a performance trap from our backlog or your own experience?All contributions will be published with attribution, helping to expand the community’s collective knowledge and supporting the development of future training materials on software performance best practices.A recording of this session is available on YouTube: https://youtu.be/iaHKeJU92Wo

Full text

The Reasonable Performance Computing SIG Knowledge Base Robert Chisholm, Peter Heywood The right tool for the job? Typical research code author ●Domain expert ○They know lots about their field ●Self-taught programmers ○Bad habits ●Short of time ○Just need the code to work ●Often working with inherited code ○May not be familiar with entire codebase �� 🔬 This leads to coding traps ●Often unnoticed in code thatʼs in use for many years by many users ●Common traps found across vastly different projects ●Potentially unreasonably poor performance Managing uniques with an array Most common mistake. With enough data 1000x speedup possible. Rather than a set Just use HPC! ●It will run faster on HPC, right? ○There can be friction getting started with HPC ○Single-threaded performance may be slower on HPC ○Often challenging to utilise parallel, distributed or GPUs Gemini 2.5 Flash FFEA - A recent case study (26th August) ●Fluctuating Finite Element Analysis (Molecular Modelling Software) ●C++ & OpenMP ●Development began 2010 ●9 authors ○PhD Students ○Postdoc Research Associates ●~12 publications Representative of much research software https://ffea.bitbucket.io/ FFEA - A recent case study (26th August) ●Asked to review itʼs performance ●Sent an example of a current userʼs workload. ●Models a bundle of “rods” ./ffea myofilaments.ffea Runtime 312 seconds FFEA - A recent case study (26th August) ●Re-compile for gprof (basic C/C++ profiler) cmake -DCMAKE_C_FLAGS=-pg -DCMAKE_CXX_FLAGS=-pg -DCMAKE_EXE_LINKER_FLAGS=-pg -DCMAKE_SHARED_LINKER_FLAGS=-pg -DUSE_OPENMP=OFF .. cmake --build . ●Then profile ./ffea myofilaments.ffea gprof ffea gmon.out > analysis.txt FFEA - A recent case study (26th August) The results: Flat profile: Each sample counts as 0.01 seconds. % cumulative self self total time seconds seconds calls ms/call ms/call name 53.94 2.12 2.12 36497000 0.00 0.00 rod::Rod::Rod(rod::Rod const&) 19.85 2.90 0.78 72994000 0.00 0.00 std::vector<...>* std::__do_uninit_copy<...>(...) 11.45 3.35 0.45 36497000 0.00 0.00 rod::Rod::~Rod() 2.29 3.44 0.09 20715750 0.00 0.00 rod::get_element_midpoint(...) 1.78 3.51 0.07 18073000 0.00 0.00 rod::Rod::get_p(int, std::array<float, 3ul>&, bool) ... FFEA - A recent case study (26th August) The significance: ●This mistake was introduced in February 2018 ○Identified 7.5 years later ●About an hourʼs work to identify and address ○It took me longer to workout how to run the example ●Central to all simulations using Rods ○10x speedup can be assumed broad ●The full simulation previously took “a week” to run. ○Now less than 11 hours. How can we enable researchers to catch and address similar problems sooner? Develop Training ●Profiling & Optimisation (Python) ○Carpentries style short-course ○Introduced here last year ○Now in beta status! ○Maintained by Jost Migenda (KCL) and myself https://github.com/carpentries-incubator/pando-python https://doi.org/10.5281/zenodo.16902755 Develop Training - Doesnʼt Scale ●It took a month to develop less than a dayʼs worth of training. ○More time then spent refining/updating it with feedback. ●It only covers the most general Python ○We stopped teaching C++ at Sheffield because there wasnʼt enough interest. ●Itʼs not possible to create a bespoke training for every combination of languages and libraries used in research. SIG-RPC Knowledge Base https://sig-rpc.github.io/ ●Mini guides ○Profiler how-toʼs ○Performance patterns ●Quick to write ●Easy to understand* *in theory Profilers ●Short high-level profiling intro ●Filtered by ○Language ○“Style” ●Suggested Sections: ○Quickstart ○Interpreting output ○Limitations Optimisations ●Filtered by ○Language ○“Subcategory” ●Suggested Sections: ○Description ○Example benchmark ○Technical Detail Easy to Maintain & Extend ●Static Jekyll website ●Guides written in markdown ○YAML header; priority, authors, name, language, style, website ○Markdown body ■<!-- more --> create the fold How can you contribute? Task 1: Reviewing Find an existing guide: ●Is it correct? ●Is it appropriate for a less technical audience? ●Are any important details missing? Create a GitHub issue/fill in a paper form. Current authors have all been highly-technical, so itʼs possible weʼve missed the mark.