Recommended approach to the application of assessment methods, and pilot applications case studies

This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101056933. Recommended approach to the application of assessment methods, and pilot applications case studies Deliverable 3.3 Authors: Carmen Gonzalez Romero (BSC), Tatiana Ferrari (DTU), Philip J McBride (DTU), Chiara Calderaro (CMCC), Simone Taddeo (CMCC), Adeola Jaiyeola (LGI), Anna Klose (Fraunhofer), Florian Eckel (Fraunhofer), Jakob Hömberg (Fraunhofer), Dana Stuparu (Deltares), Adam Jabłoński (WSB), and Andreas Villwock (HEREON) With contributions from: Fulvio Biddau (CMCC) Ref. Ares(2025)6243020 - 31/07/2025 D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 2 Document Information GRANT AGREEMENT 101056933 PROJECT TITLE Supporting and standardizing climate services in Europe and beyond PROJECT ACRONYM Climateurope2 PROJECT START DATE 01/09/2022 RELATED WORK PACKAGE WP3 RELATED TASK(S) T3.1 LEAD ORGANIZATION BSC AUTHORS Carmen Gonzalez Romero (BSC), Tatiana Ferrari (DTU), Philip J McBride (DTU), Chiara Calderaro (CMCC), Simone Taddeo (CMCC), Adeola Jaiyeola (LGI), Anna Klose (Fraunhofer), Florian Eckel (Fraunhofer), Jakob Hömberg (Fraunhofer), Dana Stuparu (Deltares), Adam Jabłoński (WSB), and Andreas Villwock (HEREON) CONTRIBUTORS Fulvio Biddau (CMCC) SUBMISSION DATE 31/07/2025 DISSEMINATION LEVEL PU-Public History DATE SUBMITTED BY REVIEWED BY VISION (NOTES) 11/07/2025 Carmen Gonzalez Asunción Lera St. Clair (BSC) First version 16/07/2025 BSC Review received 31/07/2025 Carmen Gonzalez Deliverable submitted Please cite this report as: Gonzalez Romero, C., Ferrari, T., McBride, P.J., Calderaro, C., Taddeo, S., Jaiyeola, A., Klose, A., Eckel, F., Hömberg, J., Stuparu, D., Jabłoński, A., Villwock, A. (2025). Recommended approach to the application of assessment methods, and pilot applications case studies, D3.3 of the Climateurope2 project Disclaimer: Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor the granting authority can be held responsible for them. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 3 Table of Contents 1. Background and Aim 8 2. Guidance on CS evaluation 9 2.1 Introduction 9 2.2 Key factors influencing CS values and benefits 10 2.2.1 Knowledge systems 11 2.2.2 Ecosystem of actors and co-production processes 13 2.2.3 Decision-context 16 2.2.4 Evaluation and delivery mode 18 2.2.5 Methodological approaches 21 3. Stakeholder consultation 31 3.1 Introduction and objectives 31 3.2 Methodology 32 3.3 Consultation method and processes 33 3.4 Analysis 34 3.5 Results and Discussion 36 3.5.1 Understanding and demand for CS 36 3.5.2 Providers, challenges and needs 41 3.5.3 Importance and benefits of CS 45 4. Conclusions and recommendations 53 List of tables Table 1. Summary of main methodological approaches used for valuing CS 23 Table 2. Interview participants 33 List of figures Figure 1. Summary of key factors influencing CS values and benefits 20 Figure 2. Framework steps in CS valuation 25 Figure 3. Examples of triple bottom line benefits 27 Figure 4. Phases to be followed in stakeholder consultation 32 Figure 5. Mind map illustrating the themes, sub-themes, and the links between them 35 Figure 6. The two most commonly referenced climate risks 36 D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 4 About Climateurope2 Timely delivery and effective use of climate information is fundamental for a green recovery and a resilient, climate neutral Europe, in response to climate change and variability. Climate services address this through the provision of climate information for use in decision-making to manage risks and realise opportunities. The market and needs for climate information has seen impressive progress in recent years and is expected to grow in the foreseeable future. However, the communities involved in the development and provision of climate services are often unaware of each other and lack interdisciplinary and transdisciplinary knowledge. In addition, quality assurance, relevant standards, and other forms of assurance (such as guidelines, and good practices) for climate services are lagging behind. These are needed to ensure the saliency, credibility, legitimacy, and authoritativeness of climate services, and build two-way trust between supply and demand. Climateurope2 aims to develop future equitable and quality-assured climate services to all sectors of society by: • Developing standardisation procedures for climate services. • Supporting an equitable European climate services community. • Enhancing the uptake of quality-assured climate services to support adaptation and mitigation to climate change and variability. The project will identify the support and standardisation needs of climate services, including criteria for certification and labelling, as well as the user-driven criteria needed to support climate action. This information will be used to propose a taxonomy of climate services, suggest community-based good practices and guidelines, and propose standards where possible. A large variety of activities to support the communities involved in European climate services will also be organised. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 5 Executive Summary Deliverable D3.3 of the Climateurope2 (CE2) project analyses the values and benefits associated with CS in order to provide recommendations to ensure long-term viability and realisation of the benefits of using CS. The analysis has been structured into two main parts: i) an in-depth assessment of the relevant literature identified in D3.1 together with an evaluation of the methods and tools that can be used to identify the values and benefits of CS , and ii) a series of interviews with decision-makers in the public sector to better understand how CS are perceived in practice. The in-depth assessment of literature investigated the values and benefits associated with the use of CS in different fields. The factors that influence the value of CS were grouped using the four components of CS developed in WP1 as base categories. ● Decision context. The sector in which a CS is used greatly influences how its value is perceived and assessed. This is because different sectors have distinct needs, decision timelines, risk tolerances, market structures, policy and governance structures and capacities to act on climate information. ● Ecosystems of actors and co-production processes. CS generate economic value only if users understand their risks and actively engage with the information. The risk aversion of users, with respect to management decisions, and the risk awareness of the users, with respect to climate threats, influence the value of a CS. There is a need for an increased understanding of the socio-cultural construction of user realities, their risk perception and to connect this knowledge to CS development. ● Knowledge systems. CS include a broader range of climate information and tools for decision making across different sectors. Forecasting systems and their characteristics have been considered by multiple studies, their added value is influenced by accuracy, lead time, specificity, spatial resolution, and the way weather parameters are reported. ● Delivery mode and evaluation. The value of climate information will only be realised if appropriate action is taken from the information provided. Enablers for an increased value of CS include collaboration, improved accessibility of climate information, advancements in climate science, institutional reforms, and opportunities to build trust. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 6 Methodological approaches for assessing the values and benefits of CS range from quantitative (cost-benefit analyses, simulations) to qualitative (case studies, interviews). Notably, the lack of standardised metrics and frameworks severely hampers comparison, learning, and scaling of best practices. Leading frameworks stress the importance of holistic, context-sensitive evaluation that integrates stakeholder engagement and covers economic, social, and environmental dimensions. The analysis resulting from the literature review confirms the need to consider both tangible and intangible benefits of CS, as well as continuous stakeholder engagement. This report argues for a deliberate, structured approach to CS evaluation, rooted in transparent methodology, usercentred design, and explicit articulation of both benefits and limitations. Stakeholder consultation was conducted through 16 qualitative interviews with stakeholders from cities across Europe. The interviews with decision-makers were designed to explore their perspective of the values that they experience and the benefits they would like to gain from the use of CS. ● Understanding and demand for CS. Climate risks are increasingly considered into every-day decision making; new urban developments, major infrastructure projects, and changes to zoning and land use often need to include climate screening. Water-related extremes, particularly floods and heat-related stress are perceived as most impactful. Many cities explicitly mentioned the importance of directing their efforts toward the most vulnerable segments of their population. The interviews expressed the need to better understand the uncertainty associated with climate data and how to communicate it to various stakeholders. EU climate audits, funding conditions, and rising public awareness were recognised as gradually enabling local governments to act. ● Providers, Challenges and Needs. Generally, the respondents had a preference for in-house provision of CS, rather than external sources. If this was not possible, the preferred provider is usually a public institution such as national meteorological institutes or major platforms like Climate Atlas, Copernicus, research centres or universities due to legitimacy, trust and financial reasons. None of the interviewees reported to rely on any standard to select the providers. A common barrier identified across several cities is the lack of a clear and cohesive national guidance on CS. Also, there are clear institutional and financial barriers including lack of staff and lack of capacity to effectively use the data. Several cities highlighted the need for standardisation and centralised climate data. The local knowledge D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 7 was mentioned as an essential asset to develop adaptation plans that feel realistic, actionable, and trusted by the community. ● Importance and Benefits of CS. The perceived importance of climate information services among municipalities is varied and shaped by a combination of institutional structures, political will, financial considerations, and local priorities. Cities face major challenges in the practical use of CS. The lack of standardisation, clear national guidance, and data accessibility are universal complaints. For some municipalities, climate considerations are strongly embedded into their planning processes. However, this was achieved after a long period of internal advocacy and organisational alignment. For other municipalities, financial constraints and competing priorities often take precedence over climate related projects. CS are seen as beneficial in areas lacking a strong legal or regulatory framework. Solid climate data can strengthen the rationale for climate-smart policies and infrastructure, especially where legal mandates are not yet in place. This enables evidence-based advocacy and justification of climate-related actions. Clear and relatable data can also be used to raise awareness and drive behavioural change across communities Keywords Climate services, public sector, adaptation, standards, standardisation, values, benefits, evaluation methods. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 8 1. Background and Aim Work package 3 (WP3) aims to take stock of, review, and analyse practical examples that demonstrate the economic, social, and environmental values and benefits enabled by climate services (CS). A key objective of WP3 is to emphasise how these values and benefits are, and can be, created and retained to ensure long-term viability of CS and the sustained realisation of their benefits. Additionally, WP3 also explores the patterns and taxonomies of business innovation within the domain of CS. This leads to the recommendations and guidance documents, building upon active engagement and participation of multiple stakeholders across the CS community. In this framing, task 3.1 explores the typologies of values and benefits, the methods and tools used or potentially used to determine them, and how these are codesigned with and experienced by users and communities. In addition to economic and financial values and benefits (e.g. cost saving, avoided damage, return on investments), this task will also address non-economic values and benefits (e.g. cohesion and resilience, non-instrumental and relational values). In the first deliverable, D3.1, task 3.1 conducted an overview of economic, social, and environmental values and benefits of CS across the globe by analysing peer-reviewed and grey literature. This scoping review illustrated the positive effects of CS across diverse sectors and different contexts. However, further work is needed to develop a more comprehensive understanding of the values and benefits associated with CS. Studies predominantly focus on specific case studies and lack of harmonisation, making it a challenge to generalise results for a broader understanding. The need for a harmonised valuation framework was particularly evident when evaluating the values from quantitative papers. In methodological terms, there is a range of well-established methods for computing these values and benefits. Nevertheless, studies must be more rigorous to better define the characteristics of the CS offered, the information accessed, and the local contexts. Complementarities between qualitative and quantitative methods remain underexplored, yet they have the potential to bring greater clarity to the estimation of the values and benefits of CS. The conclusions of the literature feed into an outline of a stakeholder consultation, designed to explore their perspective of the values that they experience and would like to gain from the use of CS. This current deliverable, D3.3, further explores the concept of value in relation to CS, with the aim of providing insights that can support standardisation efforts regarding the evaluation of CS. This D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 9 involves an in-depth assessment of the relevant literature identified in D3.1, aimed at uncovering the nuances underlying the values and benefits described in the literature review (whereas D3.1 addressed these values in a more general manner, emphasising what has been reported and assessed). This is complemented by one-on-one interviews with decision-makers in the public sector arena. The interviews aimed to further extend and deepen the identification of values and benefits associated with CS, and to identify some of the non-economic benefits identified by users and purveyors of CS. With this information, WP3 will provide key findings and recommendations to harmonise the design and implementation of evaluation methods. In addition, the results of Task 3.1 will complement the views obtained in Task 3.2 on business innovation. There, the perspective is more from the provider side, how the value proposition is generated taking account the user needs. Although that task focuses more on private providers of CS, for instance Deliverable 3.4 addresses business innovation supporting the adaptation processes needed for sustainable and transformative societal changes. Thus, are current CS capable of providing valuable information and advice towards the transformation to climate resilient societies? Or do we need different climate services using new innovative methods to accomplish this goal? Integrating the views of both tasks of WP3 will provide more insights and answers to these questions. 2. Guidance on CS evaluation 2.1 Introduction CS involve the generation, translation, analysis, and dissemination of climate and socioeconomic data to help decision-makers in various sectors manage risks, plan for the future, and improve resilience (Vaughan and Dessai, 2014). CS transforms complex climate information into actionable insights that are accessible and relevant to specific users (Vaughan and Dessai, 2014). In this, CS hold immense value, far beyond just producing climate information. They serve as tools that equip people, businesses, and governments to make informed decisions, enhance their preparedness, and improve efficiency in addressing climate-related risks. In today’s world, where the effects of climate change and natural variability are becoming increasingly evident (IPCC, 2018), having access to reliable and relevant CS is more important than ever. When we talk about the ‘value’ of CS, we look at how it can directly impact the ability to act, whether in the short or long term, as D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 16 There is a need for an increased understanding of the socio-cultural construction of user realities, their risk perception and their knowledge. As such, this knowledge needs to be integrated in the development, acceptability and utilisation of CS. There is also a lack of standardisation in how to contextualise CS (e.g., the types of questions to ask, types of information to collect, stakeholders to co-produce with). The results of attempts to contextualise CS are scarce and difficult to compare, suggesting a lack of shared and clear goals and interactivity. It would therefore be pertinent to establish a set of principles for effective contextualisation of CS approaches as they are potentially important to guide transformations to sustainability (Martinez et al, 2022). 2.2.3 Decision-context The sector in which a CS is used greatly influences how its value is perceived and assessed. This is because different sectors have distinct needs, decision timelines, risk tolerances, market structures, policy and governance structures and capacities to act on climate information. Additionally, the level of engagement between users and providers and their ability to co-generate tailored services may vary, which in turn affects their level of involvement and ultimately the benefits and value derived from the service. Value can be described in the form of: • Improved decision-making (e.g. when to plant crops) • Reduced economic losses (e.g. avoiding flood damage) • Enhanced resilience to climate risks (e.g. early warning systems) • Optimisation of resources (e.g. water management strategies) The CE2 report, D4.5 (Halsnæs et al. 2025), highlights distinct differences in how cities, the energy sector, and the financial sector engage with, and value CS. Cities often engage with CS to support climate adaptation planning and risk assessments, deriving value primarily through improved resilience and compliance with policy frameworks. Their benefits are often social and environmental, perceived over longer timescales. In contrast, the energy sector, especially in renewables, uses CS to manage resource variability and optimise operations, making them more sensitive to technical accuracy and operational relevance. Here, the value is more immediate and tangible, linked to efficiency and risk reduction. The financial sector, still emerging in its use of CS, seeks data-driven insights to assess climate-related risks in investments. For them, value is closely tied to quantifiable financial risk reduction and return on investment. Based on the analysis of the D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 17 three sectors in D4.5, it is clear that the perceived value and benefits of CS vary significantly. These differences in perception and use will naturally influence any attempt to assess the overall value of CS, highlighting the need for sector-specific evaluation frameworks. Considering the review presented in D3.1, it is evident that the way that the evaluation CS is produced and reported depends on the market structure and capacities to act. Most of the literature focuses on the agriculture, forestry, and fishing sectors, where the market structure relies heavily on the engagement of numerous individual actors who are often difficult to monitor and reach. Some evaluation methods in this context use simulation models that compare a baseline scenario with an ideal decision-making outcome informed by CS, assuming that all individuals adopt the recommended actions. Other studies rely on surveys to understand how users engage with CS, assess their usage patterns, and estimate the resulting benefits. However, this approach is timeconsuming and requires considerable effort to capture and synthesise the behaviour of multiple, diverse stakeholders. Often in the agriculture sector users have a relatively high capacity to act (in comparison to other sectors) on climate information, as they are typically the direct recipients of CS and must make individual decisions about whether or not to use them. This autonomy enables a more immediate link between receiving information and taking action. However, the actual capacity to act can vary significantly. Small-scale and resource-poor farmers, in particular, may face substantial constraints, including limited access to financial resources, technology, or institutional support, that hinder their ability to effectively respond to climate information (Archer et al., 2024). These socioeconomic and structural barriers can reduce the overall effectiveness and impact of CS in these contexts, despite the availability of relevant information. For example, Baffour‑Ata et al. (2024) showed that, in Ghana 82 % of cassava farmers reported receiving weather forecasts, but merely 9 % had access to sustainable agricultural technologies, and less than 14 % received support from financial, social, or governmental institutions, significantly constraining their ability to act on climate advice. In contrast, other sectors often feature a more centralised market structure with fewer, more easily identifiable actors, which can simplify the monitoring of CS use. However, in these cases, the complexity of institutional decision-making processes can become a barrier to fully capturing the value and benefits generated by CS - for instance, WMO (2024) highlights that the use of D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 18 meteorological and hydrological services, often serving sectors like aviation, transport, and disaster response operates with siloed budgets, lack of inter-agency accountability and fragmented institutional frameworks, can undermine streamlined deployment and uptake of CS, even when the information exists and the actors are easily identifiable . Thus, institutional and organisational operating regulations may also limit the ability to act on forecasts. Bureaucratic structures and regulatory frameworks can prevent even the most accurate forecasts from leading to meaningful action (Rayner et al., 2005). Jones et al. (2017) reviewed the constraints and enablers influencing the uptake of long-term climate information in sectoral investment and planning decisions, particularly in urban planning, infrastructure, and flood/coastal management. Key constraints include a disconnect between providers and users, limitations of appropriate climate data, financial and technical barriers, as well as institutional, political, and economical challenges. 2.2.4 Evaluation and delivery mode Evaluation The value of CS is directly related to the capacity of the service to satisfy the existing demand, and it is shaped by the level of interactions with end-users. Paraphrasing Hope (2015), the value of climate information will only be realised if appropriate action is taken from the information provided. Hence, the importance of fully understanding the needs and perceptions of the endusers that underline the demand of a CS, to deliver the CS in a timely manner and through the appropriate channels, and also to better understand how CS influences decision-making processes. Failing to properly identify the demand hinders the value of CS in an objective and subjective way. From an objective point of view, developing CS that are not fit for purpose implies a miscalculation of their value and a waste of resources. From a subjective point of view of the decision-maker, even when the demand (and its context and users´ perception) is comprehensively identified, there might be behavioural, financial and cultural factors that hamper the willingness and capacity to adopt actions derived from the CS. For example, farmers might not implement actions as suggested in agroclimatic bulletins because of a lack of financial resources (Born et al., 2021). Different stakeholders perceive the value of CS in varying ways, with producers and users often having distinct perspectives (Murphy, 1993). Therefore, different users apply the information based on their unique needs, constraints, and objectives, so they are influenced by decision-maker D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 19 characteristics (Murphy, 1993). Factors such as gender, socioeconomic class, social status, and education influence how individuals’ access, interpret, and apply CS (Carr & Owusu-Daaku, 2016). Other elements also contribute to varying levels of risk tolerance (see section 2.2.1 above), which in turn affects the perceived value of CS (Millner & Washington, 2011). One of the primary challenges that evaluation efforts face in these sectors is the lack of baseline data for comparing the output and outcomes of the CS (Tall et al., 2018). Whilst evaluation outputs refer to tangible elements (products or services) produced by a CS as a result of its activities, outcomes refer to the results or impacts of an evaluation process, indicating whether a CS achieved its intended goals and objectives, establishing a causal relationship between the climate service and the outcome or impact. The challenges around the outputs and outcomes are very different. The former are addressed by using indicators and frameworks that report direct numbers of application views, trainees, and any other data collection related to the success of the project or CS (Bonn et al., 2024). On the other hand, the attribution of behavioural changes or impact to outcomes of CS remains a challenge. The use of climate services happens in complex ecosystems, multilevel governance structures and dynamic cultural and social settings, making it imperative to understand the needs of the local decision-makers (including final users). Baseline data can sometimes also be difficult to obtain due to data scarcity or lack of granularity. Additionally, the inherent difficulties of monetising non-economic benefits make it more challenging to assess the benefits of CS compared to other interventions (Tall et al., 2018). Delivery mode In disaster risk management, timely delivery of the service is essential to make use of the information (Braman et al., 2013). Seydou et al. (2023) evaluated the impact of seasonal agroclimatic information used for early warning and reducing the vulnerability of farmer communities in Southwestern Niger. The study found that providing farmers with accurate and timely agroclimatic information significantly improved their ability to make informed decisions regarding crop management, planting schedules, and resource allocation and ‘led to an increase in production and reduced many forms of disaster risks, including floods and droughts’ (Seydou et al., 2023). However, delivering the service to a wider audience is crucial to make use of its full potential. By then, the service was disseminated to individuals in seminars, reaching less than 10% of the population. The authors suggested that the national meteorological service should ‘disaggregate seasonal agroclimatic information at a lower administrative subdivision to allow D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 20 community radio and mobile telephone users to disseminate them using the adapted language(s)’ (Seydou et al., 2023). Co-design or user-centred services is essential to grow the demand of CS and to increase their perceived and real value. By investing in a demand-driven approach, service providers invest more in improving the transparency, relevance, and trust, thus the quality of a CS, paving the way to a valuable and impactful CS. The use of climate information is context-dependent, meaning that the usefulness of these CS relies on the specific decision-making circumstances (adapted from Lemons et al., 2002). The value of CS can be significantly enhanced when it is tailored to meet the specific needs of the user. Hackenbruch et al. (2017) highlighted the growing importance and the need for tailored climate information to support decision-making, which must align with the context in which the user is operating. Cognitive factors as well, such as risk perception and trust in scientific sources, determine whether a decision-maker views a forecast as credible and actionable (Savari et al., 2024). Conversely, enablers include collaboration, improved accessibility of climate information, advancements in climate science, institutional reforms, and opportunities to build trust. Figure 1. Summary of key factors influencing CS values and benefits D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 21 2.2.5 Methodological approaches A crucial aspect that influences the outcome of the valuation of CS is the methodology selected to assess its values and benefits. The literature presents several distinct approaches for identifying impacts and benefits, which can be categorised into qualitative and quantitative methods, as well as ex-ante and ex-post approaches. Each method has its own strengths and weaknesses and may vary in terms of data requirements, expertise, time, and resources. In this section, we will explore the most commonly used methodological approaches in the literature to better understand their selection and the key considerations that play a crucial role in influencing the estimated value. The benefits of CS can be categorised into tangible and intangible benefits. Tangible benefits have a direct impact and can be quantified in economic or numerical terms. Examples include increased crop productivity, economic savings, and cost reduction associated with disaster preparedness. These benefits are typically assessed through quantitative methods. As highlighted in D3.1, two commonly used conceptual frameworks for evaluating the value of CS are the cost-loss approach and the cost-benefit analysis (CBA). However, both approaches have limitations, they often overlook non-quantifiable benefits, struggle with uncertainty, and may fail to capture equity and long-term societal impacts. In contrast, intangible benefits refer to outcomes that are difficult to measure but still provide significant value. These benefits can include impacts on well-being, the environment and community resilience. Lazo et al. (2008) categorised the values and benefits of CS using a triple bottom line approach, dividing them into social, environmental, and economic dimensions. While his work emphasised the importance of quantifying the value of CS, he also recognised that some benefits are difficult or impossible to assign monetary values to. Acknowledging these intangible benefits, even without financial quantification, provides a more holistic and accurate understanding of the true impact of CS. Building on this perspective, Suckall and Soares (2022), in their literature review of CS evaluations in Asia, found that current assessments are often narrowly focussed on short-term economic gains. They conclude by advocating for a broader valuation framework, one that also captures intangible and non-market benefits, ensuring a more inclusive and comprehensive evaluation of CS. D3.1 notes a growing trend in the use of qualitative approaches in the last years to understand the benefits of CS, usually made through interview methods. However, it also emphasises that many D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 22 of these qualitative studies have lacked sufficient methodological rigor and have not adequately addressed the three bottom line aspects. As stated, the estimation of values and benefits of CS can also be divided into ex-ante and ex-post approaches. An ex-ante approach is prospective and refers to the assessment of the potential value of a CS prior to implementation, often making use of historical data, simulations and models for the assessment (Meza et al., 2008). This approach is more common in quantitative analyses and is especially useful for understanding the potential impacts of a CS on specific activities, allowing for more effective resource allocation and investment. This approach can also be used to explore how further improvements to a pre-existing CS can impact economic returns and user benefits. One limitation, however, could be the oversimplification of the interaction between society and the climate (Soares et al., 2018). If users are not engaged, an ex-ante approach can overlook different social, institutional, and geographic factors that shape CS access and application, obscuring the distributional impacts of the CS. In contrast, an ex-post approach is retrospective and refers to the actual benefits that have occurred following the delivery of a CS (Meza et al., 2008). More commonly seen in qualitative analyses, this approach usually involves the analysis of observed data, user feedback, and measurable outcomes, although not necessarily seeking to quantify economic value. An ex-post approach, therefore, offers valuable insights in how CS can influence decision-making, mainly due to the involvement of users, and is useful for illustrating impact and informing future plans. However, data limitation is an issue (Soares et al., 2018), as an ex-post approach requires established monitoring and long-term data collection, which may not be feasible for some valuations. D3.1 showed that qualitative estimations of the value of CS have a wide range of approaches. Estimation approaches were classified into five broad categories that included simulation-based approaches (57%), CBA approaches (4%), econometric approaches (20%), general equilibrium models (2%) and willingness to pay (WTP) based techniques (17%). By far the most popular approach to estimating the value of CS to users, simulation approaches predominately use stochastic modelling and historical data to calculate the expected value for climate forecasts across different scenarios. Table 1 gives a brief summary of the main methods used, their classification, the relevant time for their use, their strengths and limitations, and the extent to which they are time consuming. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 23 Table 1. Summary of main methodological approaches used for valuing CS METHOD TYPE PRIMARY USE KEY STRENGTHS KEY LIMITATIONS TIME INTENSITY Cost-Loss Models Quantitative / Ex-ante Forecasting use cases Simple to apply; decision-oriented & user-focused. Ignores broader social & environmental impacts. Moderate SimulationBased Models Quantitative / Ex-ante Scenario analysis Useful for assessing potential outcomes over time; flexible. Requires detailed data inputs. High Cost-Benefit Analysis (CBA) Quantitative / Both Economic evaluation Standardised, widely accepted, & suitable for policy. Requires reliable data; undervalues intangible benefits. Moderate Willingnessto-Pay (WTP) Quantitative / Ex-ante Valuing preferences Reveals user preferences & perceived value. Subject to hypothetical bias; costly to survey. Moderate Econometric Approaches Quantitative / Ex-post Impact assessment Can control for multiple variables. Requires statistical knowledge & data. High General Equilibrium Models Quantitative / Both Policy evaluation Captures systemic interactions; useful for large-scale analysis. Highly data-intensive with complex assumptions. Very High Case Studies Qualitative / Ex-post Evaluating impact Provides in-depth, rich insights into specific contexts. Difficult to generalise; time intensive. High Interviews Qualitative / Both Stakeholder feedback Provides nuanced, context-rich data; flexible. Subjective, not easily generalisable, & requires skilled facilitation. High The lack of standardisation in the metrics used to report economic value is an issue. There are significant differences across studies, making it difficult to establish relationships or make comparisons between them. For example, one study by Mjelde et al. (1997) reported the value per area, while other studies by Mjelde, Thompson and Nixon (1996) and Mjelde et al. (1997;2) reported net profit per farm. Studies from Yu, Wang and Smith (2008), Wang et al. (2009), and Bert et al. (2006) reported gross margins per hectare per year, and studies by Nidumolu et al. (2021) and Sonka, Changnon and Hofing (1988) focussed on cost reduction per hectare per year. Additionally, studies by Quedraogo et al. (2018), Awolala et al. (2023), Amegnaglo, Mensah-Bonsu and Anaman (2022), Paparrizos et al. (2021), and Kandlikar (1998) used WTP as a metric. Thus, there is no dominant metric, leading to considerable variability across the studies and a lack of comparison among CS. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 24 In summary, the selection of an appropriate evaluation methodology for CS depends on several contextual factors, including data availability, the intended use of the evaluation, stakeholder engagement, and the type of benefits being assessed. Whilst quantitative, ex-ante methods are well-suited to forecasting economic gains, qualitative and ex-post methods are better aligned with capturing real-world impacts and intangible outcomes. However, some guidance on the economic value metrics reported should be made to enable the extrapolation of the values and allow for comparison between studies. 2.2.6 Evaluation Frameworks Several frameworks exist to guide the valuation of CS. These frameworks provide structured approaches to assess the benefits, costs, and overall value of CS in diverse contexts, helping researchers and practitioners to conduct rigorous and comprehensive analyses. In this section, we present and describe three well-established frameworks that serve as key references and practical guides for CS valuation practices. Lazo et al. (2008) - Primer on Economics for National Meteorological and Hydrological Services Lazo et al. (2008) present a decision-oriented economic framework designed to assist the National Meteorological and Hydrological Services (NMHSs) in evaluating the benefits of their services. A key contribution of their work lies in shifting the focus from purely technical assessments of CS accuracy to understanding the decision-making value these services generate. By introducing economic tools and principles, the framework provides a structured approach for assessing the societal and economic impacts of NMHSs. The ten-step framework is illustrated in Figure 2. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 25 Figure 2. Framework steps in CS valuation Source: Lazo et al., 2008 The first step aims to establish a baseline by identifying the services provided and the users of those services, followed by defining the decisions influenced by the information. Since valuation refers to comparing the value in one situation to another, the baseline sets the scenario with the outcomes associated with the status quo, that is, if nothing changes and no CS are provided. The second step involves determining the object being valued. In this step, it is crucial to describe the services provided in detail, as changes in the quality of forecasts and services can represent a marginal change in value. It is therefore important to understand exactly what is being evaluated and to describe the value chain of a CS in as much detail as possible to capture the type of information being valued. The framework offers a comprehensive method for identifying both costs and benefits, emphasising the inclusion of all impacts regardless of who incurs them or where they are realised. Considerable attention is devoted to understanding the different types of costs and benefits and how to incorporate them into the valuation process. Lazo et al. (2008) distinguishes between D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 32 user opinions, that is, we allow an open discussion covering their impressions and experiences, without guiding their responses to expected outcomes. There are four phases to implement a stakeholder consultation, as illustrated in Figure 4. Figure 4. Phases to be followed in stakeholder consultation This guideline outlines the criteria and procedures for Phase 1: Selection of stakeholders, Phase 2: Consultation methodology, and Phase 3: Conducting the consultation. The following sections describe the methodology used for the user consultation. 3.2 Methodology Stakeholder consultation was carried out using an in-depth interview approach. This method enables the collection of in-depth qualitative insights into decision-making processes, perceived challenges, and stakeholder needs related to CS. It also allows participants the flexibility to express their views freely, which helps better understand the values and benefits they expect from, or have experienced through, the use of CS. Data collection was conducted through 16 qualitative interviews with stakeholders from cities across Europe (Table XX). Participants included city managers and other relevant decision-makers occupying roles within local governments that require engagement with climate-related issues, regardless of whether they are current users of CS. The interviews were conducted online over a four-month period (February 2025 – May 2025), with each session lasting approximately 50 minutes. All interviews were recorded, transcribed verbatim, and analysed using thematic analysis. Coding and analysis were conducted in QualCoder software, following a primarily deductive coding approach informed by existing literature and research questions. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 33 Table 2. Interview participants ID INTERVIEWEE JOB TITLE COUNTRY REGION N1 Climate manager Northern Ireland Northern N2 Politician Norway Northern N3 Nature and environment specialist Denmark Northern N4 Nature and environment specialist Denmark Northern N5 Climate special consultant Denmark Northern E1 Spatial planner Slovakia Eastern E2 Vice mayor Poland Eastern E3 Head of strategic department Poland Eastern E4 Climate manager Poland Eastern S1 Project researcher Spain Southern S2 Head of water technologies unit Spain Southern S3 Civil protection Italy Southern S4 Climate change expert Italy Southern W1 Climate adaptation manager France Western W2 Ecological manager France Western W3 Climate adaptation manager and urban data manager Germany Western W4 Climate adaptation manager Germany Western 3.3 Consultation method and processes WP3 partners identified a list of potential interview participants based on personal contacts, public events organised by other European projects and networks like MIP4Adapt 1 , or the Community of Practice from projects such as Climateurope2 2 , ASPECT 3 or CLIMAAX 4 . A total of 40 local and regional governments from 11 countries across Europe were contacted via email and invited to participate in the interview. The invitation email outlined the purpose and details of the interview (see Annex 1 for reference). Of these 40 institutions, 16 accepted the invitation for an interview and to collaborate with this deliverable. The remaining 24 either did not respond to the initial or follow up emails or rejected the invitation. Once participation was confirmed, a calendar invite was shared, containing information relevant to the interview, including 1 MIP4Adapt 2 Climateurope2 3 ASPECT 4 CLIMAAX D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 34 the agreed upon date and time, and either the in-person location or a link to the virtual meeting, as per Annex 2. The interviews were led by CE2 WP3 partners in the preferred languages of the interviewees, when possible, in order to avoid potential misunderstandings or language barriers. Before beginning the interview, WP3 partners circulated the CE2 consent form to be signed by the interviewee. The consent stated that the user was freely participating in the activity, allowed for the interview to be recorded and informed the interviewee of their rights, complying with GDPR. All consent forms, recordings and interview transcripts were uploaded to Teedy, the CE2 repository designed to maintain GDPR compliance. A detailed interview protocol was developed between WP3 partners to ensure a comprehensive and accurate data collection process. The interviewer followed the protocol outlined in Annex 3. 3.4 Analysis Thematic analysis was employed as the analytical technique to identify, analyse, and report patterns and themes within the interview data (Byrne, 2022; Braun & Clarke, 2006). This method was chosen because it facilitates the systematic identification of recurring ideas and insights across qualitative data, allowing for a nuanced understanding of stakeholder perspectives. The analysis followed the six-phase approach outlined by Braun and Clarke (2006): 1) familiarisation with the data, 2) generating initial codes, 3) searching for themes, 4) reviewing themes, 5) defining and naming themes, and 6) producing the final report. This structured process ensured a rigorous and transparent approach to data interpretation. As we were working with multiple partners, we decided to use a deductive coding approach, in which codes were established a priori based on the research questions and relevant literature. In total, 21 code labels were used. A document containing the labels, definitions, descriptions, and examples was provided to the partners, who then coded the interviews using the same analytical framework. The partners systematically reviewed the data and applied the predefined codes to the relevant segments of the interviews. The third step consists of searching for themes. In this stage, the themes created are temporary and may be subject to change (Braun and Clarke, 2006). Thus, this stage was used to review the D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 35 coded segments in order to identify patterns and relationships between them. The fourth step involves reviewing the themes. Here, the data associated with each theme was read, taking into consideration if the data supported each theme or not, and if it worked within the context of the entire dataset. In the end three main groups were created. In the fifth step, we named and defined themes and created the connections with sub-themes and the mind map, shown in Figure 5. • Understanding and demand for CS: This theme aims to capture the interviewee’s knowledge and understanding of CS, what informs their usage, and what is shaping current and future demand. • Providers, Challenges and Needs: This theme focuses on understanding what data is used, including how providers are selected, as well as operational constraints and enablers. • Importance and Benefits of CS: This theme seeks to explore the importance of climate information on decision-making, and how CS is valued across economic, environmental, and social dimensions. Figure 5. Mind map illustrating the themes, sub-themes, and the links between them D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 36 3.5 Results and Discussion 3.5.1 Understanding and demand for CS The first set of questions aimed at capturing the interviewee’s knowledge and understanding of climate risks affecting their city. It explored whether, and in what ways, climate data or forecasts are used in planning and decision-making processes, their importance, and specific requirements or gaps. Dialogue with local and regional authorities revealed that two climate risks, shown in Figure 6, are particularly influential in shaping local policy and day-to-day decision-making: waterrelated extremes, particularly flooding, and heat-related stress associated with rising average temperatures and more frequent heatwaves. To account for these climate risks, certain climate variables stand out as top priorities almost everywhere. Water-related metrics such as total rainfall, rainfall intensity, storm surge potential, runoff rates, and flood extents, directly shape the design of drainage systems, green infrastructure, and coastal defences. Temperature-related variables, like daily maximums, heatwave frequency, ‘tropical nights’, and measures like the physiologically equivalent temperature that capture how heat feels, are just as crucial for cities facing hotter, longer summers. Less obvious but equally important variables such as relative humidity, wind speed, soil moisture, shading, and evapotranspiration help cities anticipate wildfire conditions, drought stress, or heat stress impacts on both people and ecosystems. Figure 6. The two most commonly referenced climate risks What emerges from the interviews is that, among all, flooding in its many forms is now the single most pressing climate threat. Coastal cities like N2, N3, and W4 reported this vividly, as they face a dangerous combination of rising sea levels and more powerful storm surges that frequently D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 37 coincide with intense rainfall events. When these hazards converge, they easily overwhelm local drainage systems and test the limits of coastal defences. As mentioned by an interviewee: But of course, in the event of heavy rainfall or in the combination of what we now have in the Baltic Sea - high tide, storm surge, risk and heavy rain - it's a completely different challenge, as we've already had situations in the past where you can simply see the water flowing directly down the street and we've already had significant infrastructure damage in various areas of the city, which is not acceptable. (W4, Germany) Flooding is not limited to the coast. Inland cities and densely urbanised regions face their own distinct flood challenges, driven by the way urban landscapes have been built over decades. Cities like W1 highlighted urban flooding as a major threat, not just because of heavier downpours but because so much of the urban fabric (concrete roads, asphalt car parks, tightly packed buildings) leaves little space for water to drain naturally. Parallel to water-related risks, heat stress has emerged as the other dominant hazard reshaping urban life. In the interviewed European cities, hotter average temperatures, prolonged heat waves, and the urban heat island effect are putting real strain on communities. Cities such as W3, which are built in valleys or bowl-like topographies, are especially vulnerable as they can trap heat and provide little natural ventilation. In southern Europe especially, local officials describe a growing problem with ‘tropical nights’, i.e. nights when temperatures remain high, instead of dropping as they once did. Alongside these two major risks, local and regional authorities also face a range of other climate hazards that are more context-specific but increasingly urgent. In southern and eastern Europe for example, prolonged droughts are threatening water supplies and agricultural production. Forest fires have been recognised to be a more and more pressing issue, not only confined to Mediterranean areas but also mentioned in northern regions, becoming a real threat to peri-urban zones as hotter, drier conditions expand fire risk. Taking into consideration all this, a clear theme across all the interviews is that what were once rare, disruptive events are now becoming part of daily reality. This means that climate risks can no longer be treated as abstract, long-term scenarios in planning documents but are increasingly integrated into everyday decision-making. EU directives and national climate laws were mentioned, as they now require local governments to develop comprehensive risk management plans, adaptation strategies, and resilience frameworks. As mentioned by N4: D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 38 Every time we make a plan, climate adaptation and flood risk reduction are a part of the planning, and that's across all the sources of water, so flood risk reduction is a part of the planning every time. I mean it's every time because we had been trying to incorporate it 10 years ago, and it took us five years to convince the entire organisation that it is a top priority. So, we are already doing it and we are very happy about that. (N4, Denmark) New urban developments, major infrastructure projects, and changes to zoning and land use often undergo formal climate screening. As mentioned by N1: A big part of where we have to consider climate data and climate risk is within our planning. Our planning teams would obviously, when they're developing planning policy and looking at spatial plans and zoning, have to think about things like flood risk. (N1, Northern Ireland) To do so, the interviewed cities have mentioned using local and publicly available data, as well as relying on partnerships with universities, research institutes, and European-funded projects. Regarding local data, regionally collected meteorological data were mentioned, particularly on temperature and precipitation. Copernicus was also mentioned as a useful resource. Integration between in-house efforts and collaborations with the research sector is a crucial element for regions to be able to successfully use projections and CS. Some examples include: We are currently working on two fronts: firstly, at the regional level to provide numerous CS, particularly through shapefiles and station data, which I primarily use for trend analysis. Secondly, we are collaborating with JASPERS [Joint Assistance to Support Projects in European Regions], a technical assistance initiative by the European Investment Bank and the European Commission. JASPERS has provided useful tools for climate verification on various infrastructures. (S4, Italy) […] the second pillar of the LIFE project within the water sector, which is an alert platform to improve flood risk management. It's called the SAT platform [Early Warning System] and is based on hydrological forecasting, aiming to reduce response times in the event of flooding. (S1, Spain) When we talk about 100-year projections and that kind of thing, we use climate scenarios, and we don’t have that many stations. (N3, Denmark) D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 39 We developed [response] plans and procedures, development of structures, in the event of floods - sewage systems, development of companies, municipal organisational units. We have a climate change adaptation plan, city development strategy until 2040, environmental protection programme, assumptions for the energy, heat and water supply plan, revitalisation programme for the city. (E4, Poland) Noticeably, many cities explicitly mentioned the importance of directing their efforts toward the most vulnerable segments of their population, incorporating specific tools and measures into their planning processes. […] we're very keen and we're doing a bit of work at the moment with UCC [University College Cork], and others, to take that climate risk data and overlay it with socio-economic data so that we can see where the most vulnerable people are and we can have that real good, useful profile both in terms of vulnerability, but I'm very keen that we look at this from a transition perspective as well, from a climate justice. So, where are the communities who are most vulnerable? We know that, we have that in Council, but what does that mean in terms of potential for heat networks? What does that mean in potential for community wealth when it comes to energy generation? (N1, Northern Ireland) Adaptation strategy developed from a vulnerability study carried out in 2014, partially updated in 2018. Currently embedded in the 2024-2030 Territorial Climate-Air-Energy Plan (PCAET), aligned with PNACC terminology. (W2, France) […] social and support policies, for example for isolated people, the elderly, and so on. We take into account the evolution, well precisely the climate. So, it's going to get hotter and hotter, for example, and so potentially we're going to need to support more and more of a population that, moreover, is aging. It's the same in the. Early childhood care policies. (W1, France) The climate analysis in the city has a multi-dimensional dimension. Issues related to air protection are the City's Strategic Priority. Improving energy efficiency is very important and revitalisation of industrial areas and industrial facilities, mining dumps create change in the city. (E3, Poland) EU climate audits, funding conditions, and rising public awareness were recognised as gradually enabling local governments to act. For example: D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 40 A big part of where we have to consider climate data and climate risk is within our planning. Our planning teams would obviously, when they're developing planning policy and looking at spatial plans and zoning, have to think about things like flood risk. (N1, Northern Ireland) Climate data and information is not informing only the policies, but some councils are beginning to assign clear budget lines for climate-related costs: Climate information is taken into account, for example, with the city budget negotiations there's always a climate component to every budget note. There are climate considerations for the approval of all kinds of projects within the city. (N5, Denmark) The way the uncertainty is handled by the different interviewees varies significantly depending on the time scale of the climate data they use. Those using climate change projections report them as highly uncertain, ‘we just call it a very wet year, because […] projections are extremely uncertain’, (N3) and more difficult to deal with than those using shorter time scales, like weather or seasonal or even historical data. The uncertainty of climate change projections is reported to have political and economic implications. On the one hand, politicians are more interested in the electoral term (typically four years) than ‘what is happening in 2050’ (S4). On the other hand, cities like N1 and the regions of S1 and S4 have reported following the ‘principle maximum caution’ when dealing with uncertainty, which is usually translated into a cost increase because you ‘need to plan for the work case scenario’ (N1). Another major concern the interviewees expressed when dealing with uncertainty is the source of the data. N3 reported challenges and concerns about how the existing different sources of climate change projections offer different outputs of sea level rise, ‘we’re talking about differences of up to one meter. I mean, that’s really significant’. The source of the data has been reported as important in order to trust the climate data and the CS provided. However, sometimes a particular set of climate data is used, even if they aren't that much trusted, ‘because it is the only one available’ (N5). There are also cities that seem comfortable when dealing with the uncertainty of the data and the models like E1, N2, N5, S1, and S3. When asked about the communication of the uncertainty associated with a CS, all interviewees agreed on the difficulty of understanding and communicating the uncertainty. Some public institutions, however, accept that uncertainty is part of working with climate data and find visual ways to represent it. Whilst others simply do not communicate the uncertainty associated with D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 41 the climate data or a climate service and just do not consider it as they ‘would not know what to do with it’ (S2). In some cases, the difficulty on communicating uncertainty rely on the time scales of the CS not matching the ‘time horizon in which humans think […] so we try not to communicate much’ (W4). The uncertainty can be a challenge to understand and communicate, but that's inherent in the science and the data itself. It's hard to get around that. (N5, Denmark) It's always important for researchers to show uncertainty […] But then for a decision maker, and for the public debate, it's often too complicated to understand because most or a lot of politicians don't understand what those numbers mean, and the public debate and the media definitely, a lot of the time do not. My experience is that it creates a lot of uncertainty and sometimes difficult political debate. (N2, Norway) Interviewees also offer a very different understanding of their concept of quality of CS. Only S1 (Spain) defined a quality CS as one that provides ‘accuracy, reliability, clarity of the results, transparency, data traceability, indices – how they are calculated, who backs the index or data.’ Trust in the source of the data is a major variable to define ‘quality’. In some instances, that level of trust is a synonym of legitimacy ‘[…] more locally accurate but also whether it comes from an organisation that is that's a good reputation and quality’ (N5) and certainty, ‘we’ve chosen to rely on specific and reliable data, even if it’s not as projected or modelled. We prefer something certain, or at least something we trust.’ (N3). 3.5.2 Providers, challenges and needs All public institutions responded that, if possible, they always prefer to follow an in-house provision of CS, rather than rely on external sources. However, they ‘rely on external providers if the data is complex’ (S2), for modelling purposes (W3), or when the CS (or part of it) is highly specialised ‘and we do not have the expertise in-house’ (S1). The preferred provider is usually a public institution like National Meteorological and Hydrological Agency or any other public agency (N1, N5, S1), major platforms like Climate Atlas or Copernicus (N3, W3), research centres or universities due to legitimacy, trust and financial reasons. None of the interviewees reported to rely on any standard to select the providers. They either follow a public bidding process or just rely on expert judgement based on proven technical expertise in the respective sectors. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 48 Another critical economic consideration is the cost of emergency response and disaster recovery. Anticipating climate events can help cities adapt and reduce the severity of their impacts. As highlighted by S1: These tools are essential. We're talking about tools that allow us to save lives, personal property, and economic assets. […] They enable us to anticipate emergencies and reduce their potential negative effects. (S1, Spain) Furthermore, CS provide an economic advantage in areas lacking a strong legal or regulatory framework at a local level. Climate information is of great value, especially where there is no legal basis yet. Because in these places, a lot of argumentations are needed, and this is better or more promising with a good data basis. (W3, Germany) Climate information is the cornerstone of a good, convincing argument. Using public EU services as a basis supports the argument further. (E1, Slovakia) Thus, having solid climate data strengthens the rationale for climate-smart policies and infrastructure, especially where legal mandates are not yet in place. It enables evidence-based advocacy and justification of climate-related expenditures. Even in day-to-day operations, some cities described that short-term forecasts help to optimise workforce planning and reduce unnecessary expenses, such as sending staff to locations where work cannot proceed due to adverse weather. Additionally, CS also offer social benefits for locations by enhancing public safety, improving communication, supporting behaviour change, and fostering inclusive planning and resilience. As climate impacts become more visible and frequent, places are increasingly turning to climate data and services not only for planning and adaptation, but also for building stronger, safer communities. One of the most vital contributions of CS to the social aspect lies in reducing risk and protecting lives (Lazo et al., 2008). Four of the interviewed locations are able to understand the contribution of CS in disaster risk reduction, especially when climate-related events such as heatwaves, floods, and storms threaten vulnerable populations. As emphasised by S3, D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 49 it's also crucial for civil defence. Comparing real-time data from a current meteorological event with climatological data allows us to identify those significant outliers that would enable us to enhance the safety level of a territory. (S3, Italy) This early recognition enhances civil defence strategies and directly contributes to the safety of a territory by preventing disasters and reducing harm during extreme events. Beyond emergency response, CS help create a well-informed society. According to N1, in addition to decision-making, climate data also supports community engagement and communication. Communication is also a benefit highlighted by E1 and W3. It also is really useful, not just in terms of decision-making, but in terms of communications, and enabling that behaviour change. So, when I go out to communities, I'm able to present data. I'm able to say, ‘look folks, in the last whatever we've seen this percentage increase in our annual average temperatures’ and it's a really good comms tool as well to allow us to do that. And to create that awareness and behavioural change that's necessary as well. (N1, Northern Ireland) By presenting clear and relatable data, municipal staff can raise awareness, encourage responsible behaviour, and drive behavioural change across communities. It also becomes a powerful tool for advocacy. Finally, in N2, the social value of CS was distilled into a single yet powerful idea: the ‘security of our society.’ Indeed, CS are not just technical tools; they are essential pillars of societal resilience, enabling municipalities to prepare, protect, and plan with people at the centre. The environmental benefits of CS have been increasingly recognised for their help on long-term sustainability, biodiversity conservation, and climate resilience. By integrating climate data into planning and decision-making, municipalities are not only adapting to a changing climate but also protecting ecosystems, improving land and water management, and transitioning toward cleaner energy systems. However, only three interviewees discussed limited environmental benefits. First, S2, where CS has assisted with the transition for renewable energy. Since the environmental department has aligned its actions with the 2030 Agenda, they use climate data to guide investment in renewable energy, particularly solar alternatives, and in this way, directly contribute to emissions reductions and cleaner urban environments. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 50 In S1, CS play a crucial role in river and land management. Authorities have used climate and hydrological data to better understand the environmental consequences of municipal interventions, such as construction in riverbeds. This awareness has prompted a shift toward more sustainable and nature-based solutions, including the designation of floodable buffer zones to manage floods in harmony with the river’s natural dynamics: It’s time to think about solutions, such as removing specific obstacles or designating floodable areas that can serve as buffer zones during floods. (S1, Spain) These solutions offer dual benefits: they reduce flood risks while supporting ecological integrity. As S1 officials emphasise, it's no longer about short-term fixes like dredging or clearing vegetation, which are costly and environmentally damaging, but about promoting long-lasting, ecologically sensitive interventions. This reflects a growing commitment to preserve biodiversity and maintain healthy river ecosystems, whilst also mitigating the risks associated with extreme weather events. E4 developed principles to manage the green urban elements of the city as well as water requirements. Furthermore, S1 acknowledges that floods are natural phenomena, not merely disruptions: […] every action undertaken by the Government of [S1] incorporates a respect for the environment and the river ecosystem and acknowledges floods as a natural part of river dynamics – they are going to happen, and they are natural events. (S1, Spain) By integrating this perspective into planning, municipalities can adopt strategies that respect natural processes rather than trying to control them entirely. CS can help operationalise this understanding by offering predictive tools and scenario planning capabilities that inform where and how to act most effectively. N4 provides a strong example of how CS can reshape urban infrastructure planning and enables municipalities to align environmental and infrastructure solutions: It would be very poor decision making if you don't have climate information because then you just do what you normally would do. And if you just do what you normally do, then you would fail. We began this transition 10 years ago and now we've been working with it more seriously about planning for five years on the stormwater side. This was because we knew that we needed to do something differently because we could see this climate information – there was a trend about thing changing, and we could not just do what we D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 51 normally did. So, I think it's very crucial that we have this information. […] right now we have this principle – one of the decisions with a wastewater management plan eight years ago is that we are trying to incorporate as many surface solutions as possible and the added value of that is that that gives something green, and more tree planting, and biodiversity back to the city instead of only working with a pipe system. That is a decision that we have been making based on this climate information, that we try to work with surface solutions instead of only piped solutions. So, we have changed our decision, our strategy, how we do it and now we can see it every time when we do a project, we have tried to do surface solutions. (N4, Denmark) Interviewees were also asked to provide concrete examples of the benefits generated through the use of CS. Below we highlight three illustrative cases that effectively summarise the diverse contributions CS can offer to local planning and decision-making, as discussed in this section: […] if we were to invest in a specific area over the next 10 years – for example, in agriculture, animal husbandry, or industry – receiving climate signals that indicate it would be counterproductive to invest in that particular area due to specific events, or that warn us about future rainfall availability or temperature fluctuations, would likely allow us to significantly limit potential damage and unnecessary projects. (S3, Italy) […] consider winter tourism. Having climatological data relevant to that sector would help us determine the long-term sustainability of installing a new ski lift. Just think in the 1970s, all the resorts in the Apennines were built with base stations around 1300 meters. Now, the effective snow elevation has risen by almost 300 meters. Resorts built in the 1970s with starting points around 1300 meters are now largely stagnant or operating at a deficit. If in the 2010s the optimal elevation was around 1300-1400 meters, it's now closer to 1600-1700 meters. This illustrates how climatological analysis could be incredibly useful even in this seemingly trivial example of tourism investment. (S3, Italy) Examples are a project called ‘living places’ to revitalise public space based on climate data, a project on renovation/relocation of public transport stops, based on the analysis of vulnerable groups and heat islands maps of ecosystem services were created based on climate data, which are used as a basis for spatial planning. (E1, Slovakia) D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 52 The interviews conducted reveal that flooding and heat stress dominate local perceptions of climate risk, shaping nearly all policy and planning priorities. Water-related climate metrics, such as rainfall intensity, storm surge, and runoff, drive drainage and flood defence designs, while temperature indicators inform heat action plans and urban adaptation. These climate threats are no longer rare; they are now part of the day-to-day operational reality for city governments, driving systematic integration of climate risk into urban planning, zoning, and budgetary decisions. Despite this progress, cities face major challenges in the practical use of CS. The lack of standardisation, clear national guidance, and data accessibility are universal complaints, as is the persistent gap between the scale of available models and the highly local detail required for realworld decisions. The complexity of climate data and uncertainty, especially in long-term projections, creates political and operational difficulties, often leaving decision-makers unsure which data to trust or how to communicate uncertainty to the public. Most municipalities prefer to rely on public or in-house data providers, though limited capacity and expertise often mean dependence on external agencies or research partnerships. Institutional and financial constraints remain critical barriers, with many cities citing insufficient staffing, lack of technical capacity, and ambiguous responsibilities as persistent obstacles. Still, some cities are moving forward by embedding climate considerations across all planning and budget lines, often motivated by EU regulations or funding requirements. Notably, municipalities are also starting to target their adaptation efforts at the most vulnerable populations, overlaying climate data with social and economic vulnerability mapping. Unlike the structured typologies proposed in Section 2, the stakeholder interviews reveal a more implicit and context-driven articulation of value. The perceived benefits of CS extend well beyond economic calculations. Municipalities report that CS enable smarter long-term investment, reduce operational and emergency costs, and support more equitable, resilient communities. Social and environmental benefits, such as improved public safety, community engagement, and biodiversity, are increasingly recognised, though often difficult to quantify and still rarely embedded into systematic valuation. Ultimately, cities see CS as essential not only for compliance or disaster response, but as foundational tools for future-proofing urban life in a changing climate. This suggests that whilst the literature provides a comprehensive taxonomy of CS value, its application in practice remains mediated by institutional constraints and subjective interpretations of what constitutes value. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 53 4. Conclusions and recommendations The literature review conducted in D3.1 focused on identifying the frameworks and methodologies to assess the value of CS described in peer reviewed papers. In contrast, D3.3 aims to identify the values and benefits that CS offers to society based on that literature review and stakeholder consultation. The values and benefits identified in D3.3 were then classified based on the four components of CS identified by the CE2 consortium. A major issue faced during the analysis of the peer-reviewed articles was that the great majority of them did not centre around operational CS (CS that were either currently, or at some point in the past, in use), but around projects, demos or pilots. This limited the possibility to identify and understand the values and benefits of the CS since the evaluation and feedback from users and providers could not be assessed, as the CS were never used or applied, especially those in Europe. In order to overcome this barrier, WP3 interviewed local decision-makers at different jurisdictional levels (small towns, cities and regions) covering Northern, Southern, Eastern and Western Europe (see Table 2 for further details). The findings of D3.3. suggest that a systematic evaluation of CS in Europe is almost inexistant and reveals a complex landscape where the ambition of evidence-based adaptation collides with significant scientific, institutional, and practical constraints. Whilst CS have become central to risk management, urban planning, and policy development, their real value is rarely straightforward, and their widespread uptake cannot be taken for granted. According to the outputs from the conducted interviews, the value of CS is determined by several interlinked factors. The quality of climate data, including accuracy, lead time, specificity, and spatial resolution, profoundly affects its usefulness. However, credibility and users’ trust are equally vital, especially given widespread preference for short-term, historical data over uncertain long-term projections. The ecosystem of actors, including risk perception, awareness, and past experiences, shapes engagement with CS, with users more likely to act if they perceive risks as relevant and urgent. Sectoral context further complicates matters: different industries and governance levels require tailored approaches, as value is perceived through varying lenses (economic, social, resilience, regulatory compliance). Methodological approaches for assessing CS value range from quantitative (cost-benefit analyses, simulations) to qualitative (case studies, interviews), with an ongoing debate D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 54 about the best balance between ex-ante and ex-post evaluations. Notably, the lack of standardised metrics and frameworks severely hampers comparison, learning, and scaling of best practices. Leading frameworks, such as those from Lazo et al. (2008), WMO (2015), and the WISER programme, stress the importance of holistic, context-sensitive evaluation that integrates stakeholder engagement and covers economic, social, and environmental dimensions. Yet, realworld practice remains inconsistent, often lacking robust monitoring, clear baselines, or systematic evaluation processes. This deliverable ultimately argues for a deliberate, structured approach to CS evaluation, rooted in transparent methodology, user-centred design, and explicit articulation of both benefits and limitations, which is essential if CS are to fulfil their promise of supporting effective climate adaptation and risk management. Key findings highlight several recurring realities: 1. Value is context-dependent and often under-realised The evidence underscores that the value of CS is not intrinsic, but highly dependent on their context of use, sectoral needs, and the decision environments of the end users. The best technical service is ultimately meaningless if it does not fit with institutional priorities, timelines, or the sociopolitical context. Many European CS described in the literature remain research-driven prototypes, with limited transition to operational services once project funding expires. The consequence is a fragmented landscape where good practice and effective services rarely scale up or endure. 2. Persistent barriers limit uptake and impact The persistent barriers to effective CS use are numerous and widely shared across sectors and regions: • Lack of standardisation: There is little harmonisation in the methods, metrics, or frameworks for evaluating CS. This makes cross-comparison or cumulative learning exceptionally difficult. Even leading frameworks (Lazo, WMO, WISER) are not consistently or rigorously implemented. • Data and capacity gaps: Many cities and regions struggle with limited access to highresolution, credible data and a lack of in-house technical capacity to process, interpret, and act upon CS. Even where technical skill exists, institutional inertia, staff shortages, and budgetary constraints are endemic. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 55 • Decision-relevance and trust: CS are often poorly tailored to the needs and capacities of real users. The mismatch between modelled data scales and real-world planning needs is acute. Trust in CS is undermined by uncertainties that are rarely communicated effectively or by conflicting data from multiple providers. • Unclear governance and fragmented provision: Many municipalities report ambiguous mandates and a lack of clear national guidance, resulting in ad hoc or inconsistent approaches to CS understanding, adoption and use. 3. Benefits go beyond economics, but remain hard to capture While economic assessments (cost-loss, CBA) are well-established, most studies continue to focus narrowly on immediate, quantifiable gains. Yet the social and environmental benefits, improved resilience, reduced vulnerability, public safety, and equity, are often the most valued by local actors, even though they are harder to monetise or systematically document. The literature and stakeholder consultations alike reveal that intangible benefits, such as increased community trust, improved communication, or better targeting of vulnerable groups, are both real and insufficiently captured by current evaluation approaches. 4. The need for genuine co-production and institutional learning Perhaps the most consistent lesson is that CS delivers greatest value when they are co-produced with, and for, their end users. Success depends not just on technical excellence but on continual engagement, capacity building, and two-way dialogue. When local knowledge and lived experience are combined with technical data, adaptation planning becomes more realistic and actionable. However, co-production is labour-intensive, requires long-term investment, and is vulnerable to shifting political and funding priorities. 5. Recommendations for a way forward • Adopt and adapt robust evaluation frameworks: Any CS initiative must be underpinned from the outset by a fit-for-purpose evaluation framework that systematically considers economic, social, and environmental benefits. While standardising metrics across Europe may be unrealistic, a common process for evaluation design, including transparent reporting of methods and assumptions, would substantially improve comparability and learning. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 56 • Invest in local capacity and data accessibility: National and European bodies must support local authorities with both technical training and the infrastructure to access, process, and use climate data at the right scale. Investments in open data, user-friendly platforms, and ongoing professional development are critical if CS are to become embedded in routine decision-making. • Prioritise communication and trust building: Clear, honest communication of uncertainty and limitations must become the norm. Without this, trust in CS will erode, regardless of technical advances. Equally, mechanisms for gathering and acting on user feedback must be routine, not exceptional. • Support institutional change and stable funding: The chronic dependence on short-term project funding undermines continuity and institutional learning. Stable, long-term financing and clear governance arrangements are prerequisites for building the institutional memory and capacity needed for CS to have enduring impact. In conclusion, the promise of CS in Europe is far from fully realised. The technical foundations exist, but their societal value remains conditional and fragile, highly dependent on user engagement, local context, and institutional commitment. Only through a sustained focus on fitfor-purpose design, rigorous and transparent evaluation, and genuine partnership with end users will CS move beyond experimental pilots to deliver on their potential as drivers of climate resilience and risk reduction. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 57 References Amegnaglo, C. J., Mensah-Bonsu, A., & Anaman, K. A. (2022). Use and economic benefits of indigenous seasonal climate forecasts: Evidence from Benin, West Africa. Climate and Development, 14(10), 909–920. https://doi.org/10.1080/17565529.2022.2047330 An-Vo, D. A., Mushtaq, S., Reardon-Smith, K., Kouadio, L., Attard, S., Cobon, D., & Stone, R. (2019). Value of seasonal forecasting for sugarcane farm irrigation planning. European Journal of Agronomy, 104, 37–48. https://doi.org/10.1016/j.eja.2019.01.005 An-Vo, D. A., Reardon-Smith, K., Mushtaq, S., Cobon, D., Kodur, S., & Stone, R. (2019). Value of seasonal climate forecasts in reducing economic losses for grazing enterprises: Charters Towers case study. The Rangeland Journal, 41(3), 165–175. https://www.publish.csiro.au/rj/RJ18004 An-Vo, D.-A., Radanielson, A. M., Mushtaq, S., Reardon-Smith, K., & Hewitt, C. (2021). A framework for assessing the value of seasonal climate forecasting in key agricultural decisions. Climate Services, 22, 100234. https://doi.org/10.1016/j.cliser.2021.100234 Archer, E., Tarhule, A., Motha, R. P., & Muchuru, S. (2024). Targeting smallholder farmers for climate information services adoption in Africa: A systematic literature review. Climate Services, 34, 100450. https://doi.org/10.1016/j.cliser.2024.100450 Awolala, D., Mutemi, J., Adefisan, E., Antwi-Agyei, P., Taylor, A., Muita, R., Bosire, E., Mutai, B., & Nkiaka, E. (2023). Economic value and latent demand for agricultural drought forecast: Emerging market for weather and climate information in Central-Southern Nigeria. Climate Risk Management, 39, 100478. https://doi.org/10.1016/j.crm.2023.100478 Baffour Ata, F., Boakye Okyere, K. A., Boafo, B. B., Ofosuhene, S. A., Tawiah, A. O., Watara, S. W., … Boakye, L. (2024). Smallholder farmers’ perceived motivations for the adoption and implementation of climate information services in the Atwima Nwabiagya District, Ghana. Climate Services, 34, 100482. https://doi.org/10.1016/j.cliser.2024.100482 Bert, F. E., Podestá, G. P., Satorre, E. H., & Messina, C. D. (2007). Use of climate information in soybean farming on the Argentinean Pampas. Climate Research, 33(2), 123–134. https://doi.org/10.3354/cr033123 Bert, F. E., Satorre, E. H., Toranzo, F. R., & Podestá, G. P. (2006). Climatic information and decisionmaking in maize crop production systems of the Argentinean Pampas. Agricultural Systems, 88(2– 3), 180–204. https://doi.org/10.1016/j.agsy.2005.03.006 Born, L., Prager, S., Ramirez-Villegas, J., & Imbach, P. (2021). A global meta-analysis of climate services and decision-making in agriculture. Climate Services, 22, 100231. https://doi.org/10.1016/j.cliser.2021.100231 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa Bruno Soares, M., & Dessai, S. (2016). Barriers and enablers to the use of seasonal climate forecasts amongst organisations in Europe. Climatic Change, 137(1–2), 89–103. https://doi.org/10.1007/s10584-016-1671-8 Bulens, J., Vullings, W., Houtkamp, J., & Vanmeulebrouk, B. (2013, May). Usability of Discovery Portals. In Proceedings of the 16th AGILE Conference on Geographic Information Science, Leuven, Belgium. D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 64 Annex 2. Template of thank you note and interview invitation [SEND WITH CALENDAR INVITE] Dear [Name], Thank you for your response and for agreeing to participate in the interview, it is much appreciated. I would like to confirm the details of our interview: Date & Time: [Confirmed date and time] Location: [If in-person, provide address; if virtual, provide meeting link] Duration: 1 hour Language: [Confirmed preferred language] Recording: [audio recorded if in person, zoom recording if virtual] I have attached a consent form to this email. It would be great if you could return the signed form to me before the interview, or during the interview if that works better for you. If you have any questions or if there are any changes to your availability, please let me know. I look forward to our discussion. Best wishes, [Name] Annex 3. Interview questions Introduction: On behalf of Climateurope2, I would like to thank you for joining us today. I am [name], I’m going to interview you, and also with us is [name] who will be assisting during our discussion. Today, we’re here to talk about climate information and data, and decision-making, and to gather your valuable insights. There are no right or wrong answers to these questions, we just want to understand your experiences. Before we begin, do you have any questions? Also, if you do have any questions at any point or want to stop, please feel free to interrupt. [If they have not yet returned the consent form, remind them to do so]. And now with your permission I will start recording and begin our interview. [wait for the interviewer’s confirmation and then start asking the questions below. Adhere to the questions and their order] 1. Factors Influencing Decision-Making a. What climate sensitivities/risks/problems do you typically face and how does this influence decision-making in your city? b. Do you take climate issues into consideration when you are making decisions? D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 65 i. How important is climate information compared to other inputs and policy priorities? c. Which time scales of climate information are most relevant to your activities? Such as short-term forecasts, seasonal outlooks, long-term projections. d. Which spatial resolutions are most relevant to your activities? e. Which climate variables are most relevant to your activities? 2. Understanding the Importance of CS a. What types of climate data and decision-making support do you consider important? b. What does the term ‘climate service’ mean to you? i. [After the response, give the CE2 definition] Climate services involve the provision of climate information in such a way as to assist decisionmaking. The service includes appropriate engagement from users and providers, is based on scientifically credible information and expertise, has an effective access mechanism and responds to user needs. So, it is not just about providing information or data but also involves assisting with knowledge development and decision making. c. After hearing the definition, are there any specific qualities or attributes that you would associate with a good climate service? d. If you use climate information or data, how do you select your provider? i. Do you process climate information in-house or rely on external providers or consultants? 1. [if they answer they rely on external providers] What type of data do you get from your provider? For what purpose? Which output do you get by using that data? 2. [if they answer they process information in-house] What type of data do you process? For what purpose? Which output do you get by using that data? ii. Do you use any quality checklist (or similar) for the climate information used in your decision-making process? e. How accurate do you think the climate information that you receive is? i. Have you experienced conflicting climate information, if so, how did you handle this? ii. Does the climate information used come with uncertainty measurement? How is it communicated [uncertainty envelopes, confidence intervals, terciles, etc.]? If not, how do you handle uncertainty? iii. Is (climate) information/data uncertainty a limiting factor in your decisionmaking processes? iv. What amount of uncertainty can you cope with in your decision-making process? D3.3 Recommended approach to the application of assessment methods and pilot applications case studies | 66 3. Values and Benefits Identified a. How valuable or beneficial do you think climate information is when it comes to decision-making? i. Can you provide some examples of the benefits created? ii. [if risk reduction is given as the only benefit] CS have been shown to have further benefits beyond risk reduction, such as greener urban spaces, improved housing, and better living conditions. Do you see any possibility of further benefits? b. What, if anything, could help you make more informed climate decisions? i. Is there any kind of climate information or improved communication that would be beneficial to your decision-making? ii. Would training or capacity building activities help you make more informed climate decisions? If so, could you please specify which trainings [the topics/elements/skills of the training] you think will help you make more informed climate decisions? 4. Barriers to Using CS a. Are there any guidelines in your sector regarding the use of climate information or data for decision-making, or is the use based on expert judgment? b. Do you face any challenges or barriers when using climate information or data for decision-making? i. Can you provide specific examples?