Heritage education under evaluation: the usefulness, efficiency and effectiveness of heritage education programmes

ARTICLE Heritage education under evaluation: the usefulness, efficiency and effectiveness of heritage education programmes Leticia Castro-Calviño 1✉, Jairo Rodríguez-Medina 2& Ramón López-Facal 1 The evaluation of programmes is necessary in order to improve their implementation and development. This paper includes the results of the evaluation of a heritage education programme entitled Patrimonializarte. It was carried out over the course of one academic year with six groups of early years, primary and secondary schoolchildren from two schools in Galicia (Spain). A holistic conception of heritage was assumed, and the evaluations of the opinion groups were gathered, analysing those of 59 direct and indirect agents (teachers, families, local heritage managers and members of the local authorities. The data were collected via questionnaires with open responses and discourse analysis was performed on the responses. The most significant results show that: (a) the teachers valued the methodology and the objectives achieved extremely positively; (b) the families particularly valued the significant learning achieved by their children regarding elements of local heritage; and (c) the local authorities and heritage managers valued the programme as it has demonstrated the need for more collaboration with schools. The programme analysed has been shown to be useful, efficient and effective and has achieved a (largely unexpected) impact on the families involved and on the local community. https://doi.org/10.1057/s41599-020-00639-z OPEN 1Department of Applied Didactics, Faculty of Science Education, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain. 2Department of Pedagogy, Faculty of Education, University of Valladolid, Campus Miguel Delibes, Paseo de Belen, 1, 47011 Valladolid, Spain. ✉email: leticiacastro. [email protected] HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z 1 1234567890():,; Introduction This paper is based on previous studies on the evaluation of the results of the heritage education programme Patrimonializarte in which the needs of teachers for working on heritage education and the learning outcomes of their pupils have been analysed (Castro-Calviño and López Facal 2019; Castro- Calviño et al., 2020a; Castro-Calviño et al., 2020b). Here, the evaluations of the teachers, families, local authorities and heritage managers are analysed. The usefulness (the achievement of objectives), efficiency (with regard to means and resources), effectiveness (higher effects to those anticipated) and impact of the programmes and the level of satisfaction of the different agents involved are evaluated. The activities developed seek to foster the creation of learning communities with the involvement of school, families and communities. The Patrimonializarte programme and its evaluation The Patrimonializarte programme was designed according to a holistic conception of heritage, which includes processes of identity (Fontal 2003). The results were analysed using a mixedmethods participatory evaluation methodology. The programme was implemented over the course of one academic year in two state schools in Galicia (Spain): one state school, which encompasses the stages of Early Childhood Education (3–6 years of age), Primary (6–12) and Secondary education (12–16) (hereafter, ECE, PE and SE, respectively) (School 1) and another school with early years and primary education (School 2). Among the participants in the research were: agents directly involved in the education process: N=111 pupils (39 from ECE, 60 from PE and 12 from SE) and N=12 teachers of different subjects; and indirect agents (external to the programme: N=43 families and N=4 members of the local authorities and heritage managers. The objective was to create a service-learning community and to involve the families and the local community in the work of the school, with the aim of fostering an atmosphere of sustainable integration for the study of the local heritage. The activities were related with the local area and mainly concerned immaterial heritage (Phase 1) and forgotten heritage (Phase 2). At the same time that the programme was implemented, an integral, integrated and integrating evaluation was carried out, taking different moments into account: evaluation of needs, initial evaluation, formative evaluation, summative evaluation and meta-evaluation (Pérez, 2002; Pérez 2006). This process was organised into two phases. In Phase 1 the needs relating to heritage education of the teachers of the two schools involved were analysed. According to the results of this analysis, the programme and the evaluation tools were designed. Subsequently, an evaluation of the design of the programme and its tools (documental composition and internal and external coherence) was carried out by a panel of external experts. With the revised programme, the initial evaluation of the pupils’ knowledge was carried out and then the programme of activities was put into action. According to Pérez (2006), during the implementation of a programme, the formative evaluation is carried out in order to analyse its execution (conformity with the planning and the existence of unexpected effects) and the framework or context (coherence with the programme and the satisfaction of the agents involved). Carrying out procedural evaluations makes it possible to verify the partial results of the programme and to change or modify any limitations (contents, methodology, timing, resources), which do not contribute towards the achievement of the proposed objectives. This evaluation requires the participation of pupils, teachers and families. In Phase 2, the summative evaluation was carried out with the participation of pupils, teachers, families and members of the local authorities and heritage managers. The aim was to verify the results of the programme as far as learning, impact and the degree of satisfaction of the agents involved are concerned. The complexity of the evaluation (objects to be evaluated, evaluating subjects, multidisciplinarity of the contents) required a planned system and the use of a mixed methodology, including both quantitative and qualitative tools (Guba and Lincoln 1989; Pérez 2006). This mixed methodology is habitual in social research in order to obtain rigorous data from different dimensions of the object of study (Johnson et al., 2007; Venkatesh et al., 2013; Onwuegbuzie and Hitchcock 2017). The importance of participatory evaluation The empowerment of the agents involved in the evaluation is an extended strategy (Fetterman and Wandersman 2007, Fetterman et al., 2015). Fetterman (2001) highlights as one of its advantages that it promotes the development of capacities, the union of the agents involved and the usefulness of the data in order to take informed decisions. This method may have a positive influence on the evaluator and on the community, thereby favouring learning processes. The inclusion of pupils, parents, teachers and local agents in the evaluation contributes towards a feeling of attachment with what is being evaluated and of empowerment in the actions which can be carried out, in this case within local heritage (Pinto and Zarbato 2017). Involving all of the interested parties in an educational programme contributes towards the dissemination of knowledge regarding the heritage studied and creates significant learning environments (Epstein, 1995; Rodríguez-Campos 2012; Marqués et al., 2020). The participation of the families assists in improving the significant learning of the pupils and improves relationships with teachers outside of the classroom (Souto-Manning and Swick 2006). A community commitment is shared, which has repercussions on the sustainability and the will to preserve heritage elements. This allows local authorities to act in accordance with these evaluations and to design or adapt heritage education programmes in order to serve local needs (Díaz-Puente et al., 2009). Different studies have shown deficiencies in heritage education on the part of local institutions, that can be mitigated with proposals, which improve the information given about heritage in their local area (Martín and Cuenca, 2011). Other deficiencies in formal education in Spain have also been identified, particularly in school curriculums, which hardly make mention of heritage education (Cuenca-López and López-Cruz, 2014; Cuenca-López et al., 2017). Thus, there is a need for working on integrated, integral and integrating heritage programmes in order to achieve significant learning. These must incorporate a rigorous and planned evaluation and should involve the whole of the educational community (Fontal and García, 2019; Fontal et al., 2019; Sánchez-Macías et al., 2019). Research aims The main objective of this study is to evaluate the quality of the Patrimonializarte programme by analysing the evaluations of the different members of the educational community who were directly or indirectly involved in the programme. The specific objectives are: SO1. To verify the usefulness, efficiency and effectiveness of the programme. SO2. To observe the level of satisfaction of the participants. SO3. To analyse the impact of the programme. ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z 2HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z Methods This research consists of an evaluative study of a group of cases employing mixed-methods (Stake 1995). The decision was taken to employ methodological complementarity with the aim of obtaining the greatest amount of data possible from the agents involved (Cook and Reichardt 1982). The combination of different qualitative and quantitative methods made it possible to identify weaknesses in some of these methods and to accommodate the multiplicity and diversity of real contexts in which the programme was carried out (Greene 2005; Johnson and Onwuegbuzie 2004; Creswell and Garrett 2008). The quantitative methodology was applied in order to evaluate the design of the programme and to analyse the pupils’learning outcomes. It was complemented by a qualitative methodology for the tools applied to the pupils and for the other agents, whose opinions are analysed in this paper: teachers, families, local authorities and heritage managers. Context and participants. For the formative and summative evaluations, an intentional sample of the agents involved in the programme (N=59) was used: the teachers (direct agents) and the families and local authorities/heritage managers (indirect agents) (Table 1). The teachers were distributed in the following way: 2 from ECE; 3 from PE and 7 from SE. In Phase 1, eleven family members were interviewed from the three educational stages: 9 of the 11 were mothers. Thirty-three families took part in Phase 2 with a balanced distribution between the two schools: only 15% of the questionnaires were completed by fathers. As far as their professional situation is concerned, the majority were in employment at the time the questionnaire was completed (85%). The level of education is more diverse and there is no clear profile, with a majority of participants having university studies (37). It should be highlighted that almost half of the families were from ECE, with the smallest group corresponding to SE. Three other people were interviewed, namely, a cultural technician, a cultural instructor and a local councillor. Tools. Different tools were designed in order to collect data from the three participating groups. For the teachers, two online questionnaires were designed with open questions: one for the formative and the other for the summative evaluation (see Supplementary Table S1 online). Written questionnaires, also with open questions, were given to the families in the two phases (see Supplementary Table S2). As far as the members of the local authorities are concerned, they filled in an online questionnaire with open questions as part of the summative evaluation (see Supplementary Table S3 online). The tools and their systems of categorisation can be seen in detail in the Supplementary Information file online. Table 2shows a summary of the categories. The data analysis was carried out with Atlas.Ti, v. 8.4.4 (1135). After open coding, based on code groundedness (Gr =number of quotations linked to a code), discourse analysis was carried out based on the system of categorisation outlined in the Supplementary Information file (Glasser and Strauss 1967). Results Teachers’opinions Formative evaluation.Intheformativeevaluationoftheprogramme, the participating teachers completed a questionnaire with open questions in which they expressed their opinions on the activities carried out, their contents and the outcomes. Figure 1summarises the joint results from the two participating schools (ten cases). In category C1: Partial achievements of the programme, a high level (FE.C1.V1: Gr =35) and an intermediate level (FE.C1.V2: Gr =4) was obtained. The teachers stated that the activities had increased their pupils’knowledge about heritage. However, in three cases, it was explicitly indicated that the knowledge acquired was not sufficient, not so much due to the activities, but because it was considered that there are contents of other subjects, which it is important to learn. In this regard, the teachers believe that the objective of increasing the pupils’awareness was achieved and that a sense of identification with their heritage was obtained: “Now they give more importance to these places, they know about their history and feel that it is important. The proof of this is that many of them go home and tell their families about what they have learned”(Quote 3:4; FE.S2.PE.T3). This indicates that the significance of the activities extends beyond the transmission of knowledge. The ten teachers evaluated positively the relationship between pupils, families and the local community and stated that there are now more families willing to collaborate in new activities. In category C2: Relevance of the contents, only a high level (FE.C2.V1: Gr =19) was detected. It was unanimously considered that the contents were adapted to each level and that the different phases of the programme were well linked. In category C3: Usefulness of the activities, a high level (FE.C3.V1: Gr =35) and an intermediate level (FE.C3.V2: Gr =3) was observed. It was considered that the activities carried out were appropriate for achieving the objective of raising awareness: “Another type of more traditional or less dynamic activity would not have succeeded in attracting the pupils’attention in thesameway”(Quote 3:12; FE.S2.PE.T3). In the intermediate level, references to setbacks concerning the chronology of the activities were found. As regards the visualisation activities, the teachers mentioned the importance of the leading role taken by the pupils, who were able to perceive that their work was important and valuable for the safeguarding of their heritage. This contributed to increasing their self-esteem and interest in heritage and its dissemination. In category C4: Appropriacy of the resources, the level of satisfaction with the resources and spaces employed was high (FE. C4.V1: Gr =30). The fact of combining classroom work with excursions and visits from family members/external agents was valued positively due to the fact that it broke up the daily routine and motivated the children, thereby enabling them to participate in the development of the activities. In this category, an intermediate level (FE.C4.V2: Gr =9) was also identified, along with a low level (FE.C4.V3: Gr =1). The main defect or problem encountered by the teachers was the difficulty of balancing the programme with other tasks, which led to a breakdown in the planned chronology. These comments were less frequent among primary school teachers. In order to resolve these deficiencies, the teachers proposed holding meetings with the teachers responsible for coordinating other school projects and organising a joint calendar in order to optimise and manage timing. Furthermore, some teachers did not feel they were able to carry out a heritage education programme on their own, declaring that they lacked training in this field. They mentioned that, without the guidance of the coordinator of the programme, it would not have been as successful as it was. Table 1 Distribution of the participants (N=59). Teachers Families Local authorities/heritage managers School 1 9 22 2 School 2 3 21 1 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z 3 In category C5: Appropriacy of the methodology, the three levels were identified, to a greater extent high (FE.C5.V1: Gr =26), followed by a Gr =2 in the intermediate level (workload and problems of coordination among teachers) and a Gr =1, low level (lack of coordination among teachers). One of the strengths mentioned by the teachers was the involvement of the different members of the school community. The motivation of the teachers was high in all levels due to the attractiveness of the activities designed. Likewise, the coordination of the programme was satisfactory. As far as the last item of the questionnaire is concerned, which was included in category C6: Degree of satisfaction, on the whole, a high level (FE.C6.V1: Gr =27) was obtained, in addition to an intermediate level in one case, who mentioned that he/she had found the involvement of teachers/pupils to be insufficient (FE. C6.V2: Gr =1). The teachers stated that the programme had contributed to their professional, and even personal, enrichment: “If the pupils learn and are motivated, I am happier in my work… any teacher would consider themselves to be professionally enriched”(Quote 5:22; FE.S1.SE.T5). “Of course I have also learned, both about heritage education and about the heritage of my town”(Quote 3:22; FE.S2.PE.T3). “It motivated me to work more in an interdisciplinary way”(Quote 7:22; FE.S1.SE.T7). The positive aspects that were most repeated by the teachers were teamwork and interdisciplinarity, understood from a transversal point of view, which contributed towards improving the relationships among pupils of different ages. Summative evaluation. The aim of the summative evaluation was to ascertain what the teachers considered to be the consequences of the programme for the participating pupils, if their attitudes towards heritage improved and if the programme had any type of Table 2 Analytical categories (the variables are not included as they are common: V1: high level; V2: intermediate level; V3: low level). Teachers Families Local authorities/heritage managers Phase 1: Formative evaluation Phase 1: Formative evaluation Phase 2: Summative evaluation C1: Partial achievements of the programme C1: Work with heritage C1: Knowledge about heritage C2: Relevance of the contents C2: Evaluation of the methodology employed C2: Attitude towards heritage C3: Usefulness of the activities C3: Family-school relationship C3: Need for programmes C4: Appropriacy of the resources C4: Relationship with external agents C4: Family-school-community relationship C5: Appropriacy of the methodology C5: Impact C5: Conservation and dissemination of the heritage C6: Degree of satisfaction Phase 2: Summative evaluation C6: school-local authorities/heritage managers relationship Phase 2: Summative evaluation C1: Usefulness of the programme C7: Management of resources C1: Usefulness of the programme C2: Effectiveness of the programme C2: Effectiveness of the programme C3: Efficiency of the programme C3: Efficiency of the programme C4: Impact of the programme C4: Impact of the programme C5: Degree of satisfaction C5: Degree of satisfaction Fig. 1 Code groundedness (Gr) in the formative evaluation by the teachers. The Gr (number of quotations linked to a code) is associated with the variables V1: High level, V2: Intermediate level, V3: Low level, for each category. ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z 4HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z repercussion or impact. Twelve teachers took part in this evaluation (Fig. 2). With regard to the implementation of the work itself, different issues were analysed, which may serve to improve future editions of the programme. In category C1: Usefulness of the programme, a high level was recorded (SE.C1.V1: Gr =56) along with an intermediate level (SE.C1.V2: Gr =4). The teachers expressed the view that the programme had contributed towards increasing their pupils’ abilities for studying heritage, although it is true that their prior level was low. They stated that heritage education is not sufficiently addressed in official curriculums and that “without initiatives such as this, extremely relevant aspects regarding heritage and the local environment are ignored”(Quote 12:11; SE. S2.SE.T2). They considered that the programme ended with a greater degree of awareness on the part of the pupils regarding the importance of preserving heritage, even in the early years of education. They highlight the fact that the pupils were able to notice the existence of material and immaterial elements which, prior to the programme, due to their familiarity, would have gone unnoticed and been undervalued. Regarding category C2: Effectiveness of the programme, the teachers awarded a high level (SE.C2.V1: Gr =18) and, in one case, a low level (SE.C2.V3: Gr =1) as it was considered that other complementary effects had not been achieved. The teachers did not detect any negative consequences for the pupils. Among the positive consequences, they highlight the fact that the pupils have developed critical thinking regarding their local heritage. First-hand knowledge of the state of preservation of their local heritage led to concern and, therefore, a greater degree of interest in the improvement thereof. Likewise, contextualising the knowledge acquired contributed towards becoming aware of the importance of material and immaterial heritage. The teachers’ evaluation is that the programme was beneficial for their pupils in all three stages of education. In addition to the increase in the pupils’knowledge and awareness regarding heritage, the teachers stated that there had been a series of unexpected improvements. They mentioned that the contents of the programme made it possible to escape from a strictly utilitarian view of knowledge, that the schoolchildren improved their time management skills, increased their social skills via the use of cooperative activities, worked on their linguistic skills, for example by carrying out personal interviews, and improved their relationships with their peers and, therefore, the atmosphere in the classroom. As far as category C3: Efficiency of the programme is concerned, the teachers attributed it with a high (SE.C3.V1: Gr =44) and an intermediate level (SE.C3.V2: Gr =18). They considered that the working group involved was sufficient for carrying out the activities. However, in some cases, it was mentioned that this aspect could be improved if the whole school (or at least more year groups) was involved. This was not because involvement was insufficient, but rather due to the benefits of opening the programme up to the whole school. Likewise, the documental composition, made by teachers for the programme guides, was well valued and it was considered that they were well used. Specifically: “They were of help in seeing the aims of the work and all the steps easily”(Quote 15:17; SE.S1.IE.T5) and “They served as a foundation and helped us to guide and begin to work in each of the phases of the programme”(Quote 18:18; SE.S2.PE.T8). It was considered that the resources employed were generally sufficient, but that more means could have been used in the exhibitions open to the public, or that more human resources could have been employed in order to create a greater repercussion. The level of involvement of the working group was well valued. The planned chronology, on the other hand, was the worst-valued aspect. This is an area for improvement mentioned in all three stages and in both schools, although this is not unanimous. Category C4: Impact of the programme, was given a high level (SE.C4.V1: Gr =49) an intermediate level (SE.C4.V2: Gr =2) and a low level (SE.C4.V3: Gr =2). Some influence on the other year groups, which were not directly involved, was perceived. The exhibitions (attended by the year groups which were not involved) were the means by which this influence was made more evident, with visits promoted by the pupils themselves. The teachers considered that it was fundamental for the pupils to make a public exhibition of their work. They recognised that it is essential for the school community to be aware of the work carried out and to contribute towards improving the self-esteem of the pupils by making their work visible and giving value to it. This activity made it possible to arouse awareness and curiosity Fig. 2 Code groundedness (Gr) in the summative evaluation by the teachers. The Gr (number of quotations linked to a code) is associated with the variables V1: High level, V2: Intermediate level, V3: Low level, for each category. HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z 5 regarding local heritage and its state of preservation and helped pupils to become accustomed to speaking in public. The teachers considered the public repercussion of the programme to be a positive aspect. They stated that its publication in the local printed press and in the digital media had led to a great turnout of the general public to the exhibition. They considered the impact of the programme to be high and stated that it has enabled a greater relationship and exchange between older people and younger generations and that families have been involved, not only in visiting the exhibition, but also in active collaboration in many of the activities carried out. The responses to item 16 indicate that, in the opinion of the teachers, the programme can be applied to other schools, with it only being necessary to adapt it to the heritage reality of each place: “In any school in any place there is a history, a past, important places and some heritage element worthy of being known and valued” (Quote 18:25; SE.S2.PE.T8). In category C5: Degree of satisfaction, a high (SE.C5.V1: Gr =33) intermediate (SE.C5.V2: Gr =2) and low level (SE.C5. V3: Gr =1) was reflected. One of the direct consequences of the programme is the improvement of knowledge and of the capacity for working with heritage among the teachers involved. Only one of the twelve teachers interviewed considered that the programme had not contributed anything particularly new given that she had already worked on it. The rest expressed satisfaction with the skills acquired. The aspects mentioned were: The acquisition of knowledge about local heritage. Improvement in the handling of collaborative work and relationship with the subject itself. The learning of new activities and the quest for creative solutions to learning needs. Work on heritage from the early years of education. The use of tools to transform the children into researchers and to give them a leading role in the learning process. The responses to item 19 agreed unanimously in requesting new heritage education programmes along similar lines. The general opinion was that the programme was beneficial as it helped to facilitate relationships between the schools and their communities. Opinions of families Formative evaluation. Eleven families took part in the formative evaluation (Fig. 3). In category C1: Work on heritage, high (FE.C1.V1: Gr: 32) and intermediate (FE.C1.V2: Gr =12) levels were found. Item 1 asked what, for the interviewee, the most important heritage element of his/her town was. None of the families responded generically, all mentioned one specific element, which coincided with those the pupils had worked on. In item 2, six families considered that work on heritage is useful as it helps them to know what life was like for their ancestors. All of those interviewed believe that this type of work can help their children to become more aware of heritage and to value it, due to the fact that having greater knowledge of something helps people to value it more (item 3). They valued the methodology used (C2) with high (FE.C2.V1: Gr =37) and intermediate (FE.C2.V2: Gr =12) levels. With regard to the learning achieved by the children (item 4), different aspects were highlighted: “They learned to take care of the environment”;“They learned about the evolution of the town”; “They learned what life was like for their parents and grandparents”. Some also perceived that learning can extend beyond school life and stated that the children also learned that “if we all apply a little interest, very beautiful things can be done”. As far as item 5 is concerned, a majority (9 out of 11) considered that the most effective activity was the work carried out with old photographs. Although the degree of satisfaction with the programme was high, five families proposed new activities, the other five considered that the activities were sufficient, and one did not answer the question. Category C3: Family-school relationship received high (FE.C3. V1: Gr =28) and intermediate (FE.C3.V3: Gr =1) levels. The families believe that they can play a significant role in the work carried out at school (item 7). The majority believe that they can collaborate in increasing the self-esteem of the pupils: “So that they see that we are interested in what they do”(Quote 2:14; FE. S1.F2) and in the integration of the families in the work on local heritage as a possible area of joint interest: “because it is important to care for the heritage we have”(Quote 5:15; FE.S1. F5). They also expressed a willingness to collaborate with the Fig. 3 Code groundedness (Gr) in the formative evaluation by the families. The Gr (number of quotations linked to a code) is associated with the variables V1: High level, V2: Intermediate level, V3: Low level, for each category. ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z 6HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z programme by contributing information and materials (photographs, tools) or taking part in visits in the local area. As far as category C4 on the relationship with external agents is concerned, only high levels were recorded (FE.C4.V1: Gr =38). In the answers to item 9, relating to the dissemination of the pupils’work on heritage outside of the school, the idea was expressed that it could be good for the children and, also, that it helps to promote an interest in heritage, in such a way that young people can participate with pride in its dissemination. They state that it is important to collaborate with the organisms responsible for managing and preserving heritage: incorporating good informative signposting in the heritage elements, promoting guided visits and activities to raise awareness. All of the families considered that the children, with their work, can contribute towards maintaining traditions, avoiding uncivil behaviour and attracting visitors to the local area (item 11). The families awarded a high (FE.C5.V1: Gr =44) and intermediate (FE.C5.V2: Gr =6) level to category C5: Impact. The ideas transmitted by the children in their homes regarding the activities they had done at school were varied, although they coincide in two aspects. On the one hand, it was stated that their children told them that they had learned many things that they had not known previously and, on the other hand, that it had been very interesting for them. However, one case stated that his/ her child had been bored at times (items 12 and 13). By evaluating the exhibition as a space of union between families and school (item 14), those interviewed appreciated this activity positively. Four out of the eleven families demanded more spaces relating to heritage education in which they could participate. Others proposed the gathering of traditional materials, such as songs, or interviewing older people regarding traditional trades, which have either disappeared or been transformed. Summative evaluation. In this phase, the overall evaluation of the families with regard to the programme and its results was gathered. Thirty-three families took part in this survey (Fig. 4). The distribution by school is balanced, although it is slightly greater in School 2 (52%, compared with 48% in School 1). From the point of view of the parents, the direct results on the pupils have been positive. Category C1, Usefulness of the programme: a high level was recorded (SE.C1.V1: Gr =266), to a much greater extent than an intermediate (SE.C1.V2: Gr =6) and low level (SE.C1.V3: Gr =28). Ninety-seven percent of the sample considered that their children had learned to value heritage to a greater extent. They stated that the children had learned things which they had not known before, that they were capable of realising the value of their history, mentioning that they had shown more interest in these issues after the programme had been carried out. Ninetyseven percent also considered that the pupils increased their abilities to recognise heritage. They stated that, in general, the children are more observant than before and tell their families what they have learned at school. They also recognised that the children ask questions about heritage and that old elements havebeguntoarousetheirattention.Thesamepercentageof participants stated that the children now show an interest in heritage. They perceive that their children have a greater interest in it and ask more about it, particularly regarding matters which are new to them, and that they talk more at home about what they have learned from the activities and visits. Another outcome unrelated to the content of the programme has been appreciated. The children have improved their relationships with other people, particularly with their classmates. One of the main strengths highlighted is teamwork. Fifty-two percent of the parents are convinced that their children’s relationships with other people are better following the programme: “It helped my child to open up to new people”(Quote 40:14; SE.S2.F29). “She considers that she has things in common with the people in the town”(Quote 44:14; SE.S2.F33). “He likes working with older children”(Quote 42:14; SE.S2. F31). “Before, he did not talk to me about the other children in the class or about what they did”(Quote 17:14; SE.S1.F6). As far as the intermediate and low level in this category is concerned, 32% of the families considered that their child’s relationships had not improved as they were already good. Sixteen percent did not know or did not answer the question. Fig. 4 Code groundedness (Gr) in the summative evaluation by the families. The Gr (number of quotations linked to a code) is associated with the variables V1: High level, V2: Intermediate level, V3: Low level, for each category. HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z 7 Category C2: Effectiveness of the programme has a high level (SE.C2.V1: Gr =112), with a minority of intermediate level (SE. C2.V2: Gr =1). In items 6 and 7, benefits such as the importance of teamwork, the motivation to learn, relationships with other groups and generations and that the programme has helped to awaken an interest in heritage were highlighted. Also mentioned were unexpected effects such as: “It taught my child how to carry out research”(Quote 20:19; SE.S1.F9); “It helped in my child’s personal and cultural development”(Quote 27:20; SE.S1.F16); and “It is a different way of building a society”(Quote 44:20; SE. S2.F33). The families awarded a high level (SE.C3.V1: Gr =45) to category C3: Efficiency of the programme. In their answers to item 8, they make reference to a good working atmosphere: “A good level of coordination and collaboration can be perceived” (Quote 19:23; SE.S1.F8) and “this has been seen in the results and in the work they have done”(Quote 32:22; SE.S2.F21). As far as the intermediate level is concerned (SE.C2.V2: Gr =4), reference was made to the resources: “With more resources and time, this work could be extended”(Quote 24:25; SE.S1.F13). In category C4: Impact of the programme, as well as the direct effects on the pupils, the possible repercussion of the programme on the families, the rest of the school and the local community was evaluated. This category obtained a high (SE.C4.V1: Gr =225), intermediate (SE.C4.V2: Gr =30) and low (SE.C4. V3: Gr =17) level. The intermediate level corresponds with the need to achieve a greater dissemination of the work carried out, and to adjust the timetables of the exhibition. The low level is also related with the low degree of dissemination of the programme in the opinion of some families. In relation to the influence that the programme has had on the families (item 9), it is relevant that 38% of the fathers and mothers did not know anything about the heritage elements studied in the programme. 28% stated that they did not know any of the elements, whereas 34% claimed to already have knowledge of them. As a consequence, almost half of the families (47%) stated that they had learned about their local heritage thanks to their children. 19% stated that they had learned some things, while 34% claimed not to have learned anything because they already knew about it. In item 10, the families were asked whether they considered that the dissemination had had an impact on the general public. 73% stated that it had, and made mention of the local press, in which this repercussion was more visible. Reference was also made to the dissemination among the families, online, at the school itself and in the local community. Among the remaining 27.3%, there were those who did not answer the question and those who considered that it would be necessary to increase the dissemination for the programme in order to achieve a greater repercussion, for example among the local authorities: “The relevant authorities need to be made aware”(Quote 30:27; SE.S2.F19). In item 11, 48% of the families stated that they believe that the work on heritage had an influence on other year groups in the school and not only on those in which the activities were carried out. They explain that the influence perceived in other year groups is positive, given that the programme was spoken about in the school and other year groups expressed an interest in participating. Thirty-six percent did not know or did not answer and 16% did not believe it had had an influence. In the responses to item 12, which asks whether the programme can be adapted to other schools, 94% responded that it can, mentioning the social and educational benefits of the programme. The majority of the respondents base their argument on the fact that there is heritage to be discovered everywhere and that the implementation of the programme is a good way of learning how to value and respect it. Item 13 inquiries about the exhibition that took place. All those answering this question gave a positive evaluation of it as an enriching experience for the pupils: “It’s a way of rewarding their effort”(Quote 17:32; SE.S1.F6); “the children like the fact that others see and value their work”(Quote 24:32; SE.S1.F14); and “it helps them to feel important and to learn to speak in public”(Quote 34:26; SE.S2.F23). Even those families who were not able to attend mentioned that they heard extremely positive comments about it. There was only one negative comment relating to the timetables. With regard to category C5: Degree of satisfaction, a general evaluation of the main aspects of the programme was made. The majority of the evaluations were of a high (SE.C5.V1: Gr =172) and intermediate (SE.C5.V2: Gr =13) level. In all, 93.9% of the families considered that the participation of their children in the programme has improved their knowledge of heritage education. The remaining participants did not answer (item 14). Item 15 inquired about the usefulness in terms of their children’s learning of the school offering more heritage education programmes. All of the families were in agreement regarding the necessity of this type of programme for the education of their children. They would like this experience to be repeated, fundamentally due to three aspects: the content: “The area is rich in heritage, it forms part of our culture”(Quote 40:36; SE.S2. F29); the heritage itself: “They learn more and value the heritage more”(Quote 32:35; SE.S2.F21); and the methodology of the programme: “It’s a motivating way to learn and to raise awareness Table 3 Summary of strengths and weaknesses of the programme in the opinion of the families. Strengths Weaknesses Acquisition of knowledge Too much time spent on some activities Exhibition Short duration Art activities Few trips Trips Lack of dissemination and collaboration (local authorities, social networks, press) Participation of the children Timetable of the exhibition Improvement in the pupils’relationships with others Lack of pupils’opinions in the exhibition Increase in pupils’interest Lack of current architectonic heritage (library, public spaces) Knowledge of changes in spaces and buildings Only applied to three year groups Knowledge of immaterial heritage Increase in knowledge of ancient heritage and the history of the town Contribution to making heritage more visible and respected The results were good Increase in the value of the community ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z 8HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z of the preservation of heritage”(Quote 42:38; SE.S2.F31); “actively involving the children is more productive”(Quote 30:36; SE.S2.F19) and “there is nothing better than to learn by playing”(Quote 40:37; SE.S2.F29). The general evaluation of the programme was unanimously positive. The number of strengths mentioned by the families was three times higher than the number of weaknesses (37 to 12 respectively) (item 16) (Table 3). Opinion of members of the local authorities.Thenumberof members of the local authorities and heritage managers is limited (3), but they are included in this study as complementary sources of information from the institutions. In general, the three people surveyed were in agreement and the graph below summarises the three opinions gathered in their summative evaluation (Fig. 5). The members of the local authorities and heritage managers gave category C1: Knowledge of heritage a low level (C1.V3: Gr =3).Inresponsetoitem1,theyconsideredthatthe children had a complete lack of knowledge of their local heritage and were not aware of the importance of preserving it: “We must carry out more awareness-raising campaigns” (Quote 1:9; C2.LA1) In category C2: Attitude towards heritage, high (C2.V1:Gr =1) intermediate (C2.V2: Gr =2) and low levels (C2.V3: Gr =1) were obtained. They believe that the children were not aware of the importance of preservation (item 2), but that they did not present negative or indifferent attitudes towards heritage assets, more due to general rules of behaviour than out of a specific interest in the issue (item 3). The need for programmes (category C3), obtained a high level (C3.V1: Gr =6). The authorities consulted recognised that, in their town, there are not enough programmes of this type (item 4). In response to item 5, they considered the implementation of heritage education programmes to be a key step towards fostering a different type of behaviour, as such programmes would contribute towards increasing information, a subsequent sense of identification and an interest among the youngest members of the community in the local heritage. In category C4: Family-school-community relationship, high levels (C4.V1: Gr =4) were also obtained. In item 6, it was stated that work on heritage is an issue, which should involve the community as a whole (families, school, local people): “It is necessary to make this an issue for everybody”(Quote 2:14; C1. LA1). As for item 7, they considered that organising exhibitions of the pupils’work makes it possible to awaken the interest of the families in this issue. Furthermore, they recognise that it is a good way of learning about the children’s work. For category C5: Preservation and dissemination of heritage, high (C5.V1: Gr =5) and intermediate levels (C5.V2: Gr =1) were found. In item 8, they commented that actions of this type can indirectly contribute towards improving the preservation of existing heritage assets: “Giving value to a resource can always help to carry out work on it”(Quote 3:16; C1.LA2). One agent expressed doubt about whether it is possible to make the town more attractive for tourists (item 9). Category C6: Relationship between school-local authorities and heritage managers received high (C6.V1: Gr =1) and intermediate levels (C6.V2: Gr =5). In item 10, they recognised that there is occasional collaboration between the authorities and the school (for activities, exhibitions). In response to item 11, they demanded a greater degree of involvement with and coordination between the different local authorities in order to promote the study of the local heritage, as they perceive that teamwork would optimise efforts along these lines. In category C7: Management of resources, a high level of proposals for improvement in heritage education in the town were obtained, along with demands for the promotion of local resources (C7.V1: Gr =3). They pointed out the need for joint planning of excursions, which lead to greater knowledge of the municipal heritage, carrying out teaching projects and publications (item 12). In item 13, the main demands made have to do with financial and material aspects: “We need money, with more money we could do a lot for our heritage. We also need bibliographical material in order to obtain information”(Quote 3:21; C1.LA2). Fig. 5 Code groundedness (Gr) in the summative evaluation by the members of the local authorities and heritage managers. The Gr (number of quotations linked to a code) is associated with the variables V1: High level, V2: Intermediate level, V3: Low level, for each category. HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00639-z ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020) 7:146 | https://doi.org/10.1057/s41599-020-00639-z 9