D3.4 Expert analysis of the initial ECF

D3.4 Expert analysis of the initial ECF Funding scheme EU-H2020-Grean Deal, H2020-LC-GD-2020-3 Project ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education Project number 101036505 Project Coordinator CIEMAT, Centro de Investigaciones Energeticas, Medioambientales y Tecnologicas Start Date of the Project 01.10.2021 Duration of project 48 months Contributing WP WP3: The development of the initial ECF Tasks Task 3.4: Expert analysis of the initial ECF Dissemination Level Public Due date 2022 December 30 Submission date 2022 December 30 Responsible partner UAB Contributing organizations CIEMAT, IST, JYU, USE, MedaResearch, ISQ, Trebag Authors: Josep Espluga, Markku Lehtonen, Version 1.0 The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101036505 Ref. Ares(2022)9002964 - 29/12/2022 H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF 2 WHO WE ARE The ECF consortium consists of ten partners. The project is coordinated by Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas-CIEMAT. Name Country Logo Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas – CIEMAT ES Instituto Superior Técnico. University of Lisbon. IST PT Universidad de Sevilla USE ES University of Jyväskylä JYU FI Universitat Autònoma de Barcelona UAB ES Meda Research Ltd MedaResearch RO Instituto de Soldadura e Qualidade ISQ PT Trebag Szellemi Tulajdon Es Projektmenedzser Korlatolt Felelossegu Tarsasag TREBAG HU Smartwatt Energy Sercuces SA Smartwatt PT Que Technologies Kefalaiouchiki Etaireia QUE GR H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF 3 ABOUT THE PROJECT Through a multidisciplinary, transdisciplinary and participatory process, ECF4CLIM develops, tests and validates a European Competence Framework (ECF) for transformational change, which will empower the educational community to take action against climate change and towards sustainable development. Applying a novel hybrid participatory approach, rooted in participatory action research and citizen science, ECF4CLIM co-designs the ECF in selected schools and universities, by: 1) elaborating an initial ECF, supported by crowdsourcing of ideas and analysis of existing ECFs; 2) establishing the baseline of individual and collective competences, as well as environmental performance indicators; 3) implementing practical, replicable and context adapted technical, behavioural, and organisational interventions that foster the acquisition of competences; 4) evaluating the ability of the interventions to strengthen sustainability competences and environmental performance; and 5) validating the ECF. The proposed ECF is unique in that it encompasses the interacting STEM (Science, Technology, Engineering, and Mathematics)-related, digital and social competences, and systematically explores individual, organisational and institutional factors that enable or constrain the desired change. The novel hybrid participatory approach provides the broad educational community with: an ECF adaptable to a range of settings; new ways of collaboration between public, private and third-sector bodies; and innovative organisational models of engagement and action for sustainability (Sustainability Competence Teams and Committees). To encourage learning-by-doing, several novel tools will be co-designed with and made available to citizens, including a digital platform for crowdsourcing, IoT solutions for real-time monitoring of selected parameters, and a digital learning space. Participation of various SMEs in the consortium maximises the broad adoption and applicability of the ECF for the required transformational change towards sustainability. H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF 4 LEGAL NOTICE The sole responsibility for the content of this publication lies with the authors. It does not necessarily reflect the opinion of the European Union. Neither the CINEA nor the European Commission is responsible for any use that may be made of the information contained therein. All rights reserved; no part of this publication may be translated, reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. The quotation of those designations in whatever way does not imply the conclusion that the use of those designations is legal without the content of the owner of the trademark. 5 TABLE OF CONTENTS 1. Executive Summary ......................................................................................................................... 6 2. Objectives ......................................................................................................................................... 9 3. Methodology .................................................................................................................................... 9 3.1. Selection of the experts ....................................................................................................... 10 3.2. The Delphi procedure .......................................................................................................... 11 4. Results and discussion ................................................................................................................... 15 4.1. Expert comments on the strengths and weaknesses of the initial ECF .............................. 15 4.2. Agreements and disagreements among the experts concerning the initial ECF ................ 21 5. Conclusions .................................................................................................................................... 28 6. References ...................................................................................................................................... 32 H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 6 of 32 1. EXECUTIVE SUMMARY The objective of this deliverable is to report on the assessment by a group of experts of D3.3, “The development of an initial ECF” (Heikkinen et al. 2022), which elaborates an initial proposal of the European competence framework (ECF) for climate change and sustainability in the form of a “roadmap”. The roadmap consists of four steps, each containing a description of the essential aspects, reflective questions, enablers, and tools for overcoming the constraints to promoting sustainability in education. The experts who conducted the evaluation come from diverse disciplines, including for instance physics, management, education, and geography, and are recognised experts in the field of education for sustainability. The experts conduct professional activities related to the EU Green Deal Cluster and/or to the ECF4CLIM advisory board. The expert analysis took the form of a Delphi process consisting of two rounds. First, the experts read the document and answered a series of seven open questions. Second, a series of 20 statements representing the most salient results of the first-round analysis was extracted, and the experts were asked to rate their degree of agreement or disagreement with the statements along a Likert scale from 1 to 5. In general, the experts consider that the roadmap presented in document D3.3 contains potentially very fruitful elements, since it is well based on sound literature, takes into account the appropriate skills and competences, and seems to be a good approach towards the operationalization of its implementation in practice. The experts underline that the real challenge is implement the roadmad in different educational contexts, something for which the ECF4CLIM project will dedicate a good part of the subsequent efforts. The main results of the expert assessment can be organized according to the four steps of the roadmap of the initial ECF proposal (see D3.3, Heikkinen et al. 2022): a) Engagement The experts fully agree that it is important to reflect upon and discuss values supporting sustainability, but also about those values that support business-as-usual and unsustainable practices, structures, and mental models. They also consider that the definition of "collective competences" can sometimes be somewhat confusing, because it can refer to organisational aspects just as well as to leadership. A more explicit H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 7 of 32 and elaborate definition may help to avoid misunderstandings, and explain, for instance, that leadership and organisational issues can indeed represent two dimensions of a single concept. The experts agree that the diversity of educational contexts is sufficiently observed in the document, but only with regard to factors relating to formal education. The project should consider whether the variation in non-formal educational context or in short-term training actions should also be taken into account. Some of the experts warn that throughout the document the environmental dimension seems to prevail over the social and economic ones. Finally, there is some divergence of views among the experts concerning the idea of relying on the established sustainability concepts, policies, and experience (the Club of Rome report, the Brundtland report, the Rio Summit, etc.), to help the individuals see their role within a larger process. b) Connections The strongest agreement that experts show in this category has to do with the idea that teachers – like professionals in any other field, for that matter – tend to consider only one of the dimensions of sustainability (environmental, economic or social – the one closest to their own area of specialisation). Fostering interdisciplinarity in the educational system is difficult yet indispensable, given that the lack of interdisciplinarity is considered as a major barrier to sustainability. The notion of interdisciplinarity, although already present in the roadmap, could be more clearly incorporated and operationalised as the project moves forward. c) Visions The most widely shared idea among experts in this regard is that achieving a perfect balance between the three classic dimensions of sustainability (environmental, economic, social) is practically impossible in the educational institutions, as it is in other spheres of society. It would be better to aspire for some kind of an “Ecotopia”, which would imply a radical change in today's economic growth paradigm towards one that emphasises a new understanding of wellbeing. However, there are doubts whether this kind of work actually remains within the scope of this project, given that sustainability learning acknowledges the need for profound transformations of the socio-technical systems, possibly with a view to a transition to some kind of an Ecotopia. In their responses, the experts did not elaborate further on how to incorporate this kind of a perspective at the school level. Future scenarios grounded in sound scientific data are seen by most but not all of the experts as essential tools for describing the impact of human actions on the planet and the means that we H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 8 of 32 have available for avoiding the worst climate scenarios. Some experts suggested adopting this type of an approach and tools to promote sustainability competences at school. d) Action In terms of ‘action’, the experts agree that competences must be developed throughout the entire curriculum, which is possible only provided that an effective commitment and coordination effort on the part of the school management. Furthermore, the experts also agree that it is necessary to recognise the real-life limitations that the schools are subject to, notably the specific factors constraining possibilities of action: limited human and economic resources, bureaucratic rules, etc. So, any proposal should be adaptable to very different educational contexts. The experts mostly consider that the initial ECF proposal tends to represent a linear idea of the way in which educational schemes operate: "feel, think, and do", and that the challenge is to adopt a more cyclical approach, moving from "doing" towards "being". The cyclical approach should be encouraged. In this sense, some experts suggest that the roadmap could maybe include more competences related to evaluation and accountability, as this would underline the cyclical nature of the process, and be more consistent with the idea of sustainability. Finally, some experts suggest that the political dimension of sustainability could be more present throughout the roadmap. All these suggestions and comments will be taken into account in the subsequent modification of the initial ECF, as will the ideas arising from other debates that will be carried out with students, teachers, school administration and management, as well as other relevant stakeholders in the education sector (policymakers, NGO, etc.). H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 9 of 32 2. OBJECTIVES The objective of this deliverable is to report on the assessment by a group of experts of D3.3 (“The development of an initial ECF”), which elaborates an initial proposal of the European competence framework (ECF) for climate change and sustainability. The initial ECF is a working document that aims to offer to educational institutions and the educational community at large practical guidance and a working tool for participatory means and processes for the promotion of sustainability in diverse educational contexts. This initial proposal of ECF is structured as a roadmap and aims to provide tools for different stakeholders to map the factors that foster or enable sustainability in various educational contexts, and to overcome the constraints and to strengthen the enabling factors. The roadmap comprises four steps. The first aims at engaging people in promoting sustainability. The second is designed to deepen people’s understanding of the complexity of sustainability. The third step concentrates on envisioning alternative futures, mapping possible pathways towards sustainability, and promoting adaptability for changes and action. Finally, in the fourth step, the values and principles, the systemic understanding of sustainability, and the envisioned sustainable futures help to elaborate strategies for action. The ECF is based on results from a crowdsourcing exercise, which sought stakeholder views on the draft European Sustainability Competence Framework, elaborated by the Joint Research Center of the European Commission and published under the title “GreenComp” (Bianchi & al. 2022). The initial ECF also draws from a review and analysis of relevant literature and policy frameworks underpinning competence frameworks in other policy areas (Task 3.2), as well as on the theoretical framework of ECF4CLIM. 3. METHODOLOGY A round for commenting on the preliminary ECF by an expert panel was organized (see below). The expert analysis took the form of a Delphi process through an on-line platform consisting of two rounds. The Delphi method consists of an interactive process based on a panel of experts who answer a series of questions over several rounds, so that they can define their positions in relation to the answers given by others, thus providing a certain consensus on the topic under study. The initial idea of organising an online evaluation seminar via an internet platform such as Zoom or Teams, a Delphi panel format was finally chosen, for two reasons. Firstly, an on-line discussion would not have been well suited for the type of task that the experts would be asked to do, that is, in-depth reading and rather elaborate commenting on D3.3. Secondly, in the period in which this H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 16 of 32 "Environmental performance" is mentioned, but it is hard to see which concrete competences might be associated with this dimension. The experts propose to incorporate more competences related to evaluation and accountability, as this would make the whole process cyclical instead of linear. This would be more consistent with the principles underpinning the concept of sustainability. Often one of the problems of sustainability work is that it focuses only on the action itself: activism without critical reflection, without review, and above all without considering the ways in which any action is embedded in a larger, complex system. It is necessary to (humbly) recognise that the pursuit for sustainability is a never-ending job, and at the same time, to have a strong commitment to carry on with the work and constantly assess progress. The idea of distinguishing between individual and collective competences is very interesting. It is a good contribution. b) Do you think the educational competences discussed in the document are adequately defined? Is the terminology used clear? According to the experts ’assessment, the terminology used is clear in general, but some concerns emerge and should be taken into account. The experts recognise that the terminology is an important problem in this area, given the multiplicity of concepts which appear as similar or identical, but might not always mean the same thing. A dictionary or a glossary at the end of the document might be useful. The definition of “collective competences” seems sometimes somewhat unclear (does it refer to organisational or leadership aspects?). Some experts ask why the concept of competences is not defined in the same way as done by the GreenCoomp (and therefore by the European Commission). In the document, the description of competences is often not formulated in a precise manner but instead in a narrative way. This can make difficult its use as a tool designed to produce actual educational results. c) Do you think that the different dimensions of sustainability (environmental, economic, social) are properly taken into account? According to the experts, the environmental dimension seems to prevail over the rest (and they find it understandable due the situation we face). Social and economic dimensions are given less attention. H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 17 of 32 The action-based competences, in practice, are much more related to the availability of economic resources and to cognitive biases of the individual or the community than what is acknowledged on the paper (D3.3). The experts tend to consider that a perfect balance between the three dimensions of sustainability (environmental, economic, social) is impossible. It is better to move towards some kind of an Ecotopia (a change in the wellness concept and the economic growth paradigm). Using these dimensions makes sense, but it would be much better to mention that these are never truly separate but rather mutually interdependent dimensions. There is a risk that at school, each teacher just discusses about the dimension that is closest to her field of expertise. Interdisciplinarity should be promoted. A strong point is that the ECF proposal addresses a set of competences beyond the classic division of areas of sustainability. This facilitates the integration of the different dimensions of sustainability. d) What do you think about the measures recommended in the document to strengthen the competences? Would you suggest any other measures? According to the experts, most of the proposed measures already exist and are already applied in educational institutions. The grand challenge is how to test, review and apply them effectively. More measures can be added, depending on the specific characteristics and pedagogical methods of the educational community in question. It is not necessary to assume, as GreenComp does, that all schools can use an active methodology (although this would be ideal). It is necessary to recognise the limitations posed by the concrete realities of each educational organisation. The measures should allow connecting high-quality knowledge and information to action, helping the students and teachers to understand the importance of taking data-driven decisions and how false information can mislead the public. Some experts consider that the document does not address concrete “measures” but instead deals with "tools”, and that it is not clear whether these are tools for the project itself or for the development of competences (addressed to learners). It would therefore be better to have a series of pedagogical questions linked to the identified competences. e) What is your opinion about the ways in which the development of competences should be assessed in practice? H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 18 of 32 The experts consider the tools defined in the document as appropriate: questionnaires, audits, quality assessments, taking notes in the engagement exercises and group meetings, etc. However, the proposed ways (peer reviews, self-assessment, reflection etc) are useful for measuring individual competences, but evaluating collective competences and environmental performance needs further exploration. The use of contextualised indicators, in a similar way to those proposed for the SDGs, are suggested by some experts. The experts warn that the competences need to be developed throughout the entire curriculum and in all different subjects, which is difficult without an effective coordination within the educational institutions. This is seen as something really difficult. They also underscore the need to be practical and not overload the teachers. According to the experts, the teachers spend lot of time thinking how to assess the competences of their students to fulfil administrative requirements from the school bureaucracy. Therefore, teachers need tools that help them to do this in a practical way. f) To what extent do you think our ECF proposal is sensitive to a diversity of educational contexts? According to the experts, the differences between educational contexts are sufficiently considered in the document. But they also warn that what is proposed is suitable only for formal education. In less formal contexts or in short-term training actions, the competences may be more difficult to promote through the means suggested in the document. Moreover, diverse actors with different interest need to be considered, if sustainability is to be achieved (something that is already contemplated in the project). In any case, it is necessary to ensure that the competences are sensitive to different contexts, so a process to adapt them is needed (methodological guidelines for ensuring concreteness and contextsensitivity is suggested). g) Any other comments concerning the strengths and/or weaknesses of the document? Proposals for improvement? The experts suggest considering the following ideas to improve the initial ECF roadmap: − Conceptualise more clearly that sustainability is a collective project, instead of merely an individual and isolated endeavour. H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 19 of 32 − The idea of sustainability should be presented as something in construction, open to the participation and contributions of all involved actors. − It would be good to include a short historical dimension to the whole and also a humanist dimension in the educational curricula (the Meadows Report, the Brundtland Report, the Rio Summit, etc.), so the students and teachers feel themselves incorporated into a larger process. − The idea of interdisciplinarity should be incorporated more clearly. − It is important to reflect upon and discuss about values that support sustainability, but also about those that support business-as-usual and unsustainable actions and models. − Explore to what extent the links among competences can be further strengthened. − The proposal should include more explicitly the problem of climate change and the loss of biodiversity as the sources for the most important crisis of our times. − Future scenarios need to be considered to describe the importance of human actions and human capability to avoid the worst climate scenarios. These future scenarios should be presented as tools that science provide us to guide our actions. − The document looks like general guidelines. What will really give weight to this document is its implementation in the classrooms, and the construction of processes and mechanisms to make the educational communities more aware of their footprint, and more sustainable. − The competences should be systematised, prioritised (order and rank) and be meaningful and operational in didactic terms. The real challenge is the formulation of an operational body for the advancement and deployment of the competences. − The proposal follows a linear educational thinking scheme: feel, think, and do. Perhaps it would be useful to advance in a more cyclical manner, which would contribute to a move from the “doing” to a “being” (also following the four pillars of education proposed by the UNESCO). The sustainability education approach could go beyond the action arising from what we think and know, and move towards a state (being) in which values are problematised and actions are deeper, shared and follow cyclic spiral. Finally, one of the experts recommends a set of very concrete modifications to the document: 1. Chapter 3 (p.11): you can add a table to show meetings, workshops, number of participants, to highlight how much effort you put into it. A table is more direct, H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 20 of 32 whereas now the reader has to read the entire 60-page document to grasp that as many as 500 people were reached and engaged. 2. Changing the number of chapters. The executive summary is not numbered. Then you could start with: i. (A short) 1. Introduction ii. Etc. iii. Methodology is reiterated here and there in the text. This generates redundancy. 3. Page 9: you say “climate mitigation actions”. What about climate adaptation? 4. Definition of competence from Oxford Dictionary refers more to competency (US use). Please look at “Sustainability Competences” (Bianchi, 2020) for definitions and distinctions. I suggest you adopt the definition used there, which follows the one adopted by the European Commission. Since your great work is based on GreenComp, you could use the same definition, for the sake of coherence. 5. Definition and title of collective competences: in GreenComp, for example, collective action refers to actions and initiatives taken by a group, community, neighbourhood, etc. Here, sometimes you refer to “collective competences” in your examples and definitions as institutional competences, collective (community) competences, or leadership (especially on page 19). Perhaps you could refine this and change to “organisational competences” or something similar to avoid confusion. 6. Pag.17 a) Figure 2: futures, I suggest you aed another projected circle to the lenses of the telescope to convey that “futures” are plural. b) The third step does not contain the concept of exploratory thinking (creativity, interdisciplinarity). If this was made on purpose, then ok. Otherwise, I suggest adding a line. c) Fourth step: I believe you meant not to include “It is important []” in the bullet point, just a stylistic overview. 7. Pag.22, 24 and questions on pages 26-27 - for value reflection: it is important to reflect and discuss on values that support sustainability, but also on those that support business-as-usual and unsustainable actions and models. H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 21 of 32 8. Pag.42, among enablers, why not adopt a systemic view to imagine alternative scenarios? I guess this could apply to all competences. 9. Pag.45 the first sentence contains clearly an overview: ¿Can we guess that nobody would like to have an ecologically sustainable future? 10. Pag.45 footnote 11: this is too important to announce it in a footnote that comes on page 45 and leave it to an Appendix. I would put it upfront. 11. In general, I suggest you make this document more concise and with operational information upfront. This is the value added of your project, a participatory approach to operationalise a CF. 12. I suggest verifying if links among competences can be further strengthened. 13. Overall nice job!!! I see a high potential in this project, mostly by catering GC to user needs. 4.2. Agreements and disagreements among the experts concerning the initial ECF The second round of the Delphi procedure consisted of evaluating a series of statements obtained from the first round (the content of which was presented in the previous section). According to the results obtained in the analysis of the second-round questionnaire, hereby we presented the main agreements about the statements. Code: 1Strongly agree 2Agree 3Neutral 4Disagree 5Strongly disagree H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 22 of 32 a) Most consensual statements In the first place, there are three statements that are subject to a practically total consensus (table 4). Table 4: Most consensual statements A B C D E Average score Statement 15 2 1 2 2 1 1,6 Statement 18 2 2 1 2 1 1,6 Statement 8 2 1 2 3 2 2 The clearest agreement is on the idea that “competences need to be developed throughout the entire curriculum and in all different academic disciplines, and this is impossible without an effective commitment and coordination within the school or university in question” (statement 15). It is an organizational challenge. The experts also show a strong agreement with the idea that “it is important to reflect and discuss about values supporting sustainability, but also about those values that support business-as-usual and unsustainable practices, structures, and mental models” (statement 18). Finally, a third overwhelmingly shared statement is that “achieving a perfect balance between the three classic dimensions of sustainability (environmental, economic, social) is practically impossible. It would be better to aspire to some kind of an “Ecotopia”, which would imply a radical change of today's economic growth paradigm towards one that emphasises a new understanding of wellbeing” (statement 8). b) Statements widely shared but with some disagreement Another group consists of statements that are widely shared among the participating experts, but with some of them disagreeing or showing a lesser degree of agreement (table 5). H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 23 of 32 Table 5: Statements widely shared, but with some disagreement A B C D E Average score Statement 7 1 4 2 1 2 2 Statement 9 1 4 1 2 2 2 Statement 12 3 1 2 3 2 2,2 Statement 16 2 1 2 3 3 2,2 Statement 19 2 3 2 2 2 2,2 Statement 20 2 1 3 2 3 2,2 Statement 6 4 1 3 1 2 2,2 Statement 1 4 2 2 1 3 2,4 The definition of "collective competences" is somewhat confusing. It is not clear if such competences refer to organisational aspects or to leadership (statement 7). School teachers tend to consider only one of the dimensions (the one closest to their own area of specialisation). Lack of interdisciplinarity in the educational system is a barrier to sustainability. The project should think about ways of overcoming this barrier (statement 9). Therefore, the idea of interdisciplinarity should be more clearly incorporated in the project (statement 19). It should not be assumed that all schools can use an active methodology (although this is ideal). It is necessary to recognise the real-life limitations, notably the specific factors that constrain the possibilities of action in any given school or university (statement 12). The different educational contexts are sufficiently observed in the document, but only those relating to formal education. Non-formal educational contexts should also be taken into account (statement 16). Future scenarios grounded in sound scientific data are essential tools for describing the impact of our actions and the means that we have available for avoiding the worst climate scenarios (statement 20). The proposal represents a linear idea of the way in which educational schemes operate: "feel, think and do". The real challenge is to adopt a more cyclical approach, moving from "doing" to "being". This would also be better in line with the four pillars of education proposed by UNESCO (statement 6). H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 24 of 32 Throughout the document, the environmental dimension prevails over the rest (which is understandable in the situation we face), while social and economic dimensions are given less attention (statement 1). c) Statements with some disagreement and diversity in the expert views There is a set of six statements that are subject to less consensus among the participating experts. Table 6: Statements with some discrepancies between experts A B C D E Average score Statement 14 4 1 2 3 3 2,6 Statement 3 4 4 2 1 2 2,6 Statement 13 4 4 2 2 2 2,8 Statement 17 4 3 1 3 3 2,8 Statement 5 5 2 3 1 3 2,8 Statement 11 4 2 3 2 4 3 The competences should be assessed by using some of the tools defined in the document: questionnaires, environmental audits, quality assessments, surveys taking notes of the SCTs and SCCs, etc. However, they are mainly useful for measuring individual competences, whereas collective competences and environmental performance need further exploration (statement 13). Basically, the document does not propose other measures than those that already exist in educational practice (statement 11). More competences related to evaluation and accountability should be included, to underline the cyclical nature of the process. This would also be more consistent with the idea of sustainability (statement 5). The best way to assess the development of competences is through contextualised indicators similar to those proposed for the SDGs (statement 14). The political dimension remains somewhat blurred; it would need greater clarity and attention (statement 3). H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 25 of 32 It would be useful to include a short historical survey of the sustainability concepts and policies (the Club of Rome report, the Brundtland report, the Rio Summit, etc.), to help the individuals see and place themselves within a larger process (statement 17). d) The least shared statements Finally, there are three statements that are the ones that concentrate the greatest discrepancies among the experts (table 7). Table 7: Less shared statements A B C D E Average score Statement 10 3 4 3 2 4 3,2 Statement 4 5 1 4 2 4 3,2 Statement 2 4 4 5 1 4 3,6 Experts tend to disagree in considering that: The document does not mention concrete “measures” to strengthen the competences, but only “tools”. It is not clear whether these are tools for the project itself or for the competences that the students should achieve and develop (statement 10). The proposal should include more explicitly the problem of climate change and the loss of biodiversity as the most important crises of our times. (Sustainability is desirable, climate change is urgent) (statement 4). "Environmental performance" is a term widely mentioned throughout the document, but it is hard to see the concrete competences that could be associated with this issue (statement 2). e) Statements selected by the experts The experts were asked to select the 5 statements they considered key or most important for the development of the project. Among them, the results show there are 2 statements that were selected by three experts, 5 were by two experts, and 10 were mentioned by at least one expert (table 8). H2020-LC-GD-2020-3, Project 101036505, ECF4CLIM, European Competence Framework for a Low Carbon Economy and Sustainability through Education D3.4, Expert analysis of the initial ECF Page 32 of 32 7. REFERENCES Bianchi, G., Pisiotis, U., Cabrera Giraldez, M. (2022). GreenComp – The European sustainability competence framework. Bacigalupo, M., Punie, Y. (editors), EUR 30955 EN, Publications Office of the European Union, Luxembourg; doi:10.2760/13286, JRC128040. Heikkinen, H., Nokkala, T.; Lehtonen, A., Mykrä, N. (2022). The development of initial ECF. Deliverable D3.3, ECF4CLIM project, European Competence Framework for a Low Carbon Economy and Sustainability through Education.