Key Performance Indicators for Research Data Management – How to doom the NFDI for good

Master A TRIZ Liberating Structures Approach to kickstart an open and constructive dialogue 26. August 2025 CoRDI 2025 - 2nd Conference on Research Data Infrastructure Impact of RDM Tobias Hamann – NFDI4ING Task Area Frank Évariste Demandt – NFDI4ING Administrative Office Matthias Fuchs – Fachinformationsdienste (FID) /Specialised Information Services (SIS) Key Performance Indicators for Research Data Management – How to doom the NFDI for good Master What to expect from this talk: TRIZ Liberating Structures 2 Original TRIZ 26.08.2025Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. General problem Generalise Specific solution Specify General solution Use one of the 76 standard solutions Specific problem Hurdle Specific problem Avoid Specific solution Specific malicious acts Identify Destroy General malicious acts Master What to expect from this talk: 3 Original TRIZ combined with TRIZ Liberating Structures 26.08.2025Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. General malicious acts Destroy Specific solution Avoid those acts Specific malicious acts Identify malicious acts, that are performed already Specific problem Hurdle KPI collection Master Consistent inconsistency The problem with Key Performance Indicators (KPIs) 4Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. 1003 Number of software items published in OPEN source 4Software Codes published 1007 Number of events HELD 2Conference 2022 and Workshop 1009 Number of events PARTICIPATED IN 8 + 3 Different Events 2001 Number of activities for community engagement 3Bilateral exchange 2002 Number of activities for community activation 1Jarves Workshop 2003 Number of activities for furthering interconnectivity 1Jarves @ FDM Werkstatt 3001 Number of datasets provided by the consortium 2 + 1 Different datasets 3002 Number of datasets fulfilling the quality criteria established by the consortium 2Different datasets 5027 Total number of visits -Jarves (Not tracked) This needs cookies. Is this legal to track? What to track here? Commits? Releases? Repositories? What is a data set? When is a consortium engaging? When is exchange bilateral? 26.08.2025 Master Let‘s doom the NFDI KPIs for good 5Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. Competitive thinking between institutions: Instead of collaboration and joint exchange, the focus is on "better" individual performance Fear of underperformance and comparison: holding back results because they are expected to be worse in comparison to other institutions and will impact reputation negatively Expectation of growth and saturation effects: growth in KPIs may be limited to certain factors, e.g. target community size Various responsibilities during the KPI assessment: different subjective interpretations of KPIs render them inconsistent 26.08.2025 Master Competitive thinking between institutions: Instead of collaboration and joint exchange, the focus is on "better" individual performance Expectation of growth and saturation effects: growth in KPIs may be limited to certain factors, e.g. target community size Which of those are already visibly conducted? 6Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. Fear of underperformance and comparison: holding back results because they are expected to be worse in comparison to other institutions and will impact reputation negatively Various responsibilities during the KPI assessment: different subjective interpretations of KPIs render them inconsistent 26.08.2025 Master Our specific malicious acts 7Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. Distortion of scientific priorities: focus on easily measurable outputs instead of substantive quality or innovation Short-termism: preference for quick results instead of long-term basic research Superficial quality assessment: important dimensions (originality, societal impact, long-term value) are not captured Fear of underperformance and comparison: holding back results because they are expected to be worse in comparison to other institutions and will impact reputation negatively Various responsibilities during the KPI assessment: different subjective interpretations of KPIs render them inconsistent 26.08.2025 Master Short-termism: preference for quick results instead of long-term basic research Superficial quality assessment: important dimensions (originality, societal impact, long-term value) are not captured Fear of underperformance and comparison: holding back results because they are expected to be worse in comparison to other institutions and will impact reputation negatively Various responsibilities during the KPI assessment: different subjective interpretations of KPIs render them inconsistent How to avoid our specific malicious acts 8Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. Distortion of scientific priorities: focus on easily measurable outputs instead of substantive quality or innovation Possible solutions: •Create KPIs for quality of research •Communicate scientific quality not only by publications but also on corresponding data sets •Choose KPIs that account for new findings and methods applied in research 26.08.2025 Master Distortion of scientific priorities: focus on easily measurable outputs instead of substantive quality or innovation Short-termism: preference for quick results instead of long-term basic research Superficial quality assessment: important dimensions (originality, societal impact, long-term value) are not captured Fear of underperformance and comparison: holding back results because they are expected to be worse in comparison to other institutions and will impact reputation negatively How to avoid our specific malicious acts 9Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. Various responsibilities during the KPI assessment: different subjective interpretations of KPIs render them inconsistent Possible solutions: •Create a culture of trusting cooperation between the consortia by utilizing various formats •Townhall •Workshops focusing on indicators •Contributions from experts •Sub-networks for exchange (if this is not initially possible in the large group) •Responsible persons work across committees 26.08.2025 Master Any Questions? 16Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. 26.08.2025 Master We need friction! 17 We do not want to abolish KPIs but make them better for everyone. To smooth something out, friction is needed. This friction is supposed to happen in discussiond here at the CoRDI and beyond! Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. 26.08.2025 Master Thank you for your attention! 18 Contact: Tobias Hamann WZL of the RWTH Aachen University ORCiD: 0000-0002-8021-5524 [email protected] +49 241 80-25855 Find this presentation on Zenodo: Reflecting on KPIs| Impact of RDM| CoRDI 2025 - 2nd Conference on Research Data Infrastructure | Hamann et al. Find the abstract of this presentation on Zenodo: Contributors: Évariste Demandt NFDI4ING Administrative Office ORCiD: 0000-0002-5705-0071 Matthias Fuchs Fachinformationsdienst Mobilitätsund Verkehrsforschung ORCiD: 0000-0003-3365-3158 26.08.2025