CoARA Action Plan

CoARA Action Plan by the Leibniz Association Status as of 11 December 2025 2 Impressum Publisher President of the Leibniz Association Martina Brockmeier Chausseestraße 111 10115 Berlin [email protected] Licence notice All text in this publication, except for quotations and unless otherwise stated, is licensed under the Attribution 4.0 International (CC BY 4.0) licence. The full text of the licence can be found at https://creativecommons.org/licenses/by/4.0/ Editorial Team Leibniz Headquarters Contact person at the Leibniz Headquarters Claudia Labisch Rue du Trône 98, 4° B-1050 Brussels [email protected] Webseite https://www.leibniz-gemeinschaft.de/ueber-uns/europa-buero DOI 10.5281/zenodo.17914756 Citation Leibniz Association. (2025). CoARA Action Plan. Zenodo. https://doi.org/10.5281/zenodo.17914756 3 Introduction – About the Leibniz Association The Leibniz Association connects 96 independent German research institutes that range in focus from natural, engineering and environmental sciences to economics, spatial and social sciences and the humanities. Leibniz institutes address issues of social, economic and ecological relevance. They conduct basic and applied research, maintain scientific infrastructure, and provide research-based services. Communicating the knowledge gained by Leibniz institutes to industry, policymakers and society is a key aspect of the Leibniz Association’s strategic objectives. The Leibniz institutes employ around 21,300 people, including 12,200 re-searchers. The evaluation procedures established within the Leibniz Association address different levels. At the institutes’ level, the Leibniz Senate’s Evaluation Procedure ascertains whether they still fulfil the prerequisites of supra-regional importance and national scientific interest in order to continue to receive joint funding by the German Federal Government and the Länder. If an institution seeks admission to the Leibniz Association or a Leibniz Institute aims at a large-scale strategic expansion, the Leibniz Association with its Procedure for Admissions and Enlargements assesses such projects in terms of their strategic benefit for the Leibniz Association and their institutional fit.1 At project level, the Leibniz Competition funding programmes address the strategic goals of the Leibniz Association as part of the national Joint Initiative for Research and Innovation. Proposals are evaluated according to their innovative strength, scientific excellence and programme-specific evaluation criteria. 1.1. The Leibniz Senate’s Evaluation Procedure The core budget of Leibniz institutes is jointly funded by the German Federal government and the Länder (German states) governments. Their Joint Science Conference (GWK) revises regularly for each institute whether the conditions for joint funding are still met. Eligibility for funding is assessed every seven years at the latest, but can be carried out earlier. The decision of the Joint Science Conference is based on an evaluation carried out by the Leibniz Senate and the independent Senate Evaluation Committee (SAE). Members of the Leibniz Senate include major German science organisations and a science organisation from other European countries. Members of the SAE are scientists who do not work at Leibniz institutes, as well as members of the Senate and representatives of the federal and state governments. Neither Leibniz institutes, the Leibniz President or any other member of the Executive Board have voting rights on either committee. The evaluation procedure comprises two basic procedural steps: (a) assessment of the structures and performance of a Leibniz institute by an evaluation group with external experts selected specifically for the institute on the basis of written documents of the institute and an on-site visit by the evaluation group to the institute, (b) followed by a 1 In this particular procedure, it is the German Science and Humanities Council that asses the scientific quality of such proposals (see 1.3). 4 recommendation from the Senate Evaluation Committee and the Leibniz Senate to the federal and state governments with a recommendation for institutional funding. The Senate Evaluation Committee prepared a self-assessment report on its procedure in which it also developed perspectives for the further design of the procedure. The Leibniz Senate agreed to it on March 19, 2024. This is referred to below. (The self-assessment report is only published in German, see https://www.leibniz-gemeinschaft.de/fileadmin/user_upload/Bilder_und_Downloads/%C3%9Cber_uns/Evaluierung/Bericht_Senat_GWK_2016-2023.pdf). 1.2. Leibniz Competition The Leibniz Association uses the internal competition in line with the research policy objectives of the Joint Initiative for Research and Innovation. With its programmes Leibniz Collaborative Excellence, Leibniz Research Alliances and Leibniz ScienceCampi the competition sets incentives for the achievement of excellent research results in particularly innovative and synergistic collaborative projects, for the development of new fields of research and for the establishment of strategically relevant focal points. The recruitment of excellent women for scientific leadership positions is the focus of the Leibniz Programme for Women Professors, whereas Leibniz Junior Research Groups enable early independence of researchers. The programme LeibnizTransfer strengthens the transfer of research results to industry and society and encourages participative formats. Two senate committees are responsible for the selection procedures, the Senate Competition Committee (SAW) and the Senate Strategic Committee (SAS). Both Committees are composed of external experts, representatives of the Leibniz Association as well as federal and Länder ministries. The projects are assessed by the external scientific experts of the respective committee. Only they have the right to vote on funding decisions in the SAW. Two Vice Presidents of the Leibniz Association hold the chair and deputy chair of the SAW, but do not have a vote in funding decisions. The president of the Leibniz Association is not a member of the SAW. The names of the members of these Committees are publicly available on the Leibniz Association’s website. The Leibniz competition funds projects with a volume of up to 1,2 million EUR, a volume of about 32 million p. a. EUR is currently available for this purpose. 1.3. The Procedure for Admissions and Enlargements If either an institution seeks admission to the Leibniz Association or an existing Leibniz Institute seeks a large-scale strategic expansion2 of its institutional funding, it is the federal and state governments that initiate a particular assessment procedure. In this 2 More than € 5 million p. a. for institutions specializing in engineering, natural sciences, biosciences or medicine or more than to € 1.5 million p. a. for institutions specializing in the humanities, social sciences and economics. 5 procedure, the Joint Science Conference (GWK) first asks the Leibniz Association to develop a position on the strategic relevance of the research field in which an application is being submitted (“research field assessment”, German: “Forschungsfeldbetrachtung”). This first step includes experts from inside and outside the Leibniz Association. The GWK then decides whether the Leibniz Association and the German Science and Humanities Council (Wissenschaftsrat) should conduct a detailed assessment procedure. In this second procedural step, a group of experts from inside and outside the Leibniz Association carries out on-site visits to the institutes concerned and assesses the strategic benefit for the Leibniz Association and the institutional fit of admissions and enlargements. Based on this they report to the Leibniz Senate, which makes a statement in view of a request for institutional funding. This part of the procedure is primarily concerned with strategic issues, while the German Science and Humanities Council assesses the scientific quality of the proposal, their supra-regional significance and their structural relevance for the science system. In the case of small-scale strategic expansion projects3, the SAS prioritizes the projects and reports to the GWK, which makes the final decision as part of the budget preparation process, considering the assessment by the SAS. 2. The reform journey so far The Leibniz Association signed The Agreement on Reforming Research Assessment (ARRA) in November 2022, thereby committing itself to implement ten core commitments. The Association is represented by three of its institutes in the following CoARA working groups: • Towards Transformations: Transdisciplinarity, Applied/Practice-based research and impact, • Responsible Metrics and Indicators. The Leibniz Association participates in the National Chapter Germany. The overarching aim is to support the implementation of the research assessment reform in Germany and to function as a forum for the discussion and coordination of CoARA matters specific to the German research landscape. In addition, members of the Leibniz Association contribute to a sub-group of the Alliance of Science Organisations in Germany focused on “Reputation and Incentives” and to a UNESCO working group on “Open Science Funding and Incentives“. In July 2024, the Leibniz Association also signed the “Barcelona Declaration on Open Research Information”. These are all commitments to research assessment, as well as to the necessary changes in the way research information is handled, and to these changes themselves. (e. g. openness as the default, and support of sustainable infrastructures for open research information). 3 Up to € 4 million p. a. for institutions specializing in engineering, natural sciences, biosciences or medicine or up to € 1 million p. a. for institutions specializing in the humanities, social sciences and economics. 6 The Leibniz Association combines research, research infrastructures and transfer in many of its institutes. In the scientific and science policy discourse, it is committed to ensuring that these elements of successful scientific work are appropriately recognised and that evaluations are not inadmissibly limited to research in the narrower sense. In November 2022, the Leibniz Association launched the initiative “Strengthening Current Research Information Systems (CRIS) in the Institutes of the Leibniz Association”. This initiative supports and promotes the widespread use of CRIS within the Leibniz Association in order to contribute to the ongoing further qualification of research information management. As all research assessment is based on research information (be it qualitative or quantitative), this is a crucial prerequisite for fair and comprehensible research evaluation processes. More information on the initiative is available in this published poster: https://doi.org/10.5281/zenodo.10961102. In October 2023, the Leibniz Association launched its reform process with a Leibniz leadership lecture that addressed existing and alternative framework conditions and questions of research evaluation in the Leibniz Association. In December 2023, the Leibniz Strategy Forum on Research Assessment was established with the aim of bringing together the various activities related to research assessment within and outside the Leibniz Association in which Leibniz representatives are involved. This allows for impulses from internal reflection processes to be incorporated into the reform process at European level and, vice versa, impulses from the European discussion to be incorporated into the internal reflection processes. The Leibniz Association will report on progress during the internal reform process and in line with the timeline established in the ARRA. 3. Action plan according to Leibniz evaluation procedures Within the framework of the above-mentioned evaluation processes, the Leibniz Association will address the core commitments arising from ARRA and present the status already achieved and potential further developments where these are already evident. The Leibniz Association is constantly reflecting on its research evaluation procedures and has already achieved some success. No journal or publication-based metrics are required, and rankings or ratings are not listed in the guidelines of the Leibniz evaluation procedures. The following commitments are therefore commented on: 1. Recognise the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research 2. Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators 5. Commit resources to the reform of research assessment as necessary to achieve the organisational changes committed to 6. Review and develop research assessment criteria, tools and processes 7 7. Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use 8. Exchange practices and experiences to enable mutual learning within and beyond the Coalition 9. Communicate progress made on adherence to the principles and implementation of the Commitments 10. Evaluate practices, criteria and tools based on solid evidence and the state-of-theart in research on research, and make data openly available for evidence gathering and research 3.1. The Leibniz Senate’s Evaluation Procedure Commitment 1 – Status achieved: • Leibniz institutes are measured by how they conceptualise and implement the three task types "research", "development and operation of research infrastructures" and "transfer". With procedural changes, this triad of tasks has been emphasised more strongly since 2018. • Leibniz institutions are measured by the extent to which the areas of responsibility are also reflected in the promotion of academic staff. • Leibniz institutions are assessed by evaluation groups with experts selected specifically for the institute. Commitment 1 – Planned further development: • The Leibniz Senate plans to ask the Leibniz institutes to provide even clearer information on the areas of "research infrastructures" and "transfer" in future, in order to highlight these tasks even more. Commitment 2 – Status achieved: • The evaluation of the Leibniz institutes is carried out as an informed peer review. Qualitative and quantitative information from the institutes serve as the basis for an assessment of the institutes' concepts, structures and performance. • Since 2018, Leibniz institutes have been asked to name ten examples of results in "research", "research infrastructures" and "transfer" that they consider to be outstanding. This has systematised the previously requested qualitative information and brought it more to the fore. • Quantitative indicators are also requested for the three task areas. 8 Commitment 2 – Planned further development: • The principle of naming "highlights" should also be extended to other subject areas of the evaluation (e.g. acquisition of third-party funding) in addition to the quantitative indicators. Commitment 5, 6,7, 10 – Status achieved: • Since 2014, all experts involved in the evaluation of Leibniz institutes have had the opportunity to provide feedback on the procedure via an online questionnaire after the procedure has been completed. • Since 2016, all Leibniz institutes have had this opportunity. • At the end of each seven-year evaluation cycle, the Leibniz Senate and the Senate Evaluation Committee reflect on the evaluation procedure and develop prospects for its further development. This involves reflecting on changes in the self-regulation of science (e.g. changes in the publication system) and other changes in the environment. In the March 2024 report, the above-mentioned questionnaires were analysed for the first time and taken into account in the development of prospects for the procedure. Commitment 5, 6, 7, 10 – Planned further development: • On 19 March 2024, the Leibniz Senate adopted the latest self-assessment report of the Senate Evaluation Committee on its evaluation procedure and plans to implement the prospects developed there in the procedural principles in autumn 2024. • The Leibniz Senate report of March 2024 states that the Senate Evaluation Committee and the Senate will take up the results of the CoARA process relevant to the evaluation procedure. Commitment 8 – Status achieved: • Members of the Leibniz Senate include major German scientific organisations and a scientific organisation from other European countries. Scientific members of the Senate and the Senate Evaluation Committee do not work at Leibniz institutions, but at universities and other non-university organisations. These networks ensure a continuous exchange. • The evaluation groups are international. This also leads to a continuous exchange of experience. • At the administrative level, the Senate Evaluation Committee is in constant exchange with various German science organisations, which either evaluate institutionally funded institutions themselves (above all: German Council of Science and Humanities) or promote extensive research networks (above all: German Research Foundation (DFG)). 9 • Other institutions orientated the development of their own system towards the Leibniz Senate's evaluation procedure (2009: Max Weber Foundation/Germany, 2010ff: Ludwig Boltzmann Society/Austria, 2016ff: National Academy of Sciences (NASU)/Ukraine). Commitment 8 – Planned further development: • Continuation of the structures that ensure an exchange. • Depending on the time budget: Expansion to an exchange at administrative level with other institutions in other European countries that evaluate institutional funding. Commitment 9 – Status achieved: • The principles of the evaluation procedure are publicly accessible (https://www.leibniz-gemeinschaft.de/ueber-uns/evaluierung/das-evaluierungsverfahren-der-leibniz-gemeinschaft). • The statements of the Leibniz Senate are published (https://www.leibniz-gemeinschaft.de/ueber-uns/evaluierung/das-evaluierungsverfahren-der-leibnizgemeinschaft). • The Senate's reports reflecting on each 7-year cycle of evaluations are published. • The procedural principles contain a set of rules for dealing with conflicts of interest. • The Leibniz institutes comment on the evaluation. The Senate Evaluation Committee and the Leibniz Senate take the Institute's comments into account when making funding recommendations to the federal and state governments. • Leibniz institutes are informed about the evaluation procedure on the basis of institute-specific requirements. • The names of the persons involved in an evaluation are published. Commitment 9 – Planned further development: • Continuation of these procedures. 3.2. Leibniz Competition Commitment 1 – Status achieved: • The Leibniz Competition explicitly mentions diverse contributions to research (for example publications, management and teaching, contributions to research infrastructures or transfer) in its programmes, proposal and CV templates and evaluation criteria. 16 4. Timetable November 2022 Signing of The Agreement on Reforming Research Assessment October 2023 Leibniz leadership lecture December 2023 Establishment of the Leibniz Strategy Forum on Research Assessment June 2024 Setting up of Working Groups of the Leibniz Strategy Forum on Research Assessment October 2024 Publication of the CoARA Action Plan by the Leibniz Association January 2025 Interim report of the Leibniz Strategy Forum on Research Assessment January 2026 Final report of the Leibniz Strategy Forum on Research Assessment From January 2026 Implementation of the revised evaluation criteria as a pilot By the end of 2027 Demonstrate progress towards reviewing, developing and evaluating criteria, tools and processes regularly