In-service teacher training in Portugal: assessing the (im)pact of the formative paradigm

15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- In-Service Teacher Training in Portugal: Assessing the (im)pact of the Formative Paradigm Educational Research Globalization, and Education of Human Beings Amélia Lopes Preciosa Fernandes Fátima Pereira Elisabete Ferreira University of Porto* Faculdade de Psicologia e Ciências da Educação da Universidade do Porto Rua do Dr. Manuel Pereira da Silva, 4200-392 Porto, Portugal +351 226 079 700 +351 226 079 725 [email protected] Introduction Seen as contributing to the transformation and improvement of the Portuguese education system by qualifying and enabling educators, continuous teacher training has been one of the main concerns of Portuguese educational policy for the past decades. In this text, we intend to present and justify the methodological design of the Impact Assessment of the Continuous Teacher Training in Portugal Project (Projecto AIF - Projecto Avaliação do Impacto da Formação Contínua de Professores em Portugal), as a first product of this project's development. This project, funded by the Ministry of Education (ME), is put through by a protocol established between the ScientificPedagogical Council for Continuous Training (CCPFC - Conselho Científico-Pedagógico da Formação Contínua) of Teachers and Educators and the University of Porto (Portugal), Faculty of Psychology and Education. The core objective of the project is to assess the impact of the system of continuous education in place since 1992, using the frame of reference approved for that assessment in * Center of Research and Intervention in Education (CIIE)/Subunit School, Curriculum and Identities Formation (ECFI)/ Faculty of Psychology and Educational Sciences. 1 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- 1999 by the ME and the CCPFC. This framework is grounded in the goals guiding the organization of continuous training, as expressed in the legal text: to improve the quality of teaching and learning; to ameliorate teachers’ competencies in the various domains of the educational activity; to encourage self-training, and practices of research and innovation; to enable the acquisition of capacities favouring the construction of school autonomy and the respective educational projects; to stimulate transformation processes at the school level capable of generating training dynamics; to support professional mobility, career reconversion and qualification supplementing programs. Fifteen years after it started to be implemented, illuminating research has not yet been conducted regarding the evaluation of its effects on the practices of teachers and on school life. The AIF project intends to fill this absence. In this paper, we begin by giving an account of the lines guiding the assessment process; then we present the options underlying the research design and their justification, as well as some of the guiding principles of the data collection instruments. Guidelines of the assessment process At the heart of impact assessment is the "appraisal of the congruence between the proposed ends and priorities and the methods and procedures used in the realized training", or, to put it in a more concrete way, the "appraisal of the obtained results, translated in the effects of training on teachers and on the life of schools, as institutions susceptible to expressing qualitative changes" (CCPFC, 1999). However, seeking to instantiate legislative [legal] changes prospectively, assessment aims at, even if only secondarily, analysing several organizational aspects that may influence those effects. Among these, the system's organizational structure (including coordination and guiding structures - among them CCPCF itself - and the training entities) assumes particular relevance, therefore intending to analyse its scientific-pedagogical coherence, its logic of functioning and its correlative efficiency and realized activity. In the reference framework (CCPFC, 1999) it is suggested that the assessment should be made through: (1) the analysis of legal documents, recommendations, and 2 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- previous studies; (2) case studies; (3) the self-evaluation protocols from training entities, to be followed by an external evaluation using sampling procedures; and (4) collecting the opinions (using questionnaires and interviews) of (i) central and regional administrations, (ii) the teachers involved in the training actions, (iii) trainers and (iv) schools of all the different educational levels in which the trainees act as professionals. In order to carry out this huge task in a profound, valid and representative manner, as far as possible, it was considered that the construction of the research design is to be regarded as a first step of the development of the project itself. For that, we took into account what the reference framework suggests in terms of methodology, the main objectives of the assessment and training, the duration of the project (12 months), the organization of the system and its current configuration and also some of the evolving aspects of the system and tendencies that could be found in previously conducted studies. Some clues emerging from the document analysis The analysis of documents happens throughout the whole project, but mainly in its initial phase, intending to: sketch the 'state of the art' regarding the results from studies already conducted and the evolution of the legal, political and financial frameworks supporting the system; and characterize its current configuration in terms of its organization and the comprised and involved human and material resources. Concerning its mission, the continuous training system attempts a balanced response to three convergent intentions connected to the individual options of training, the institutional dynamics and the logic of transformation of the educational system (Santos, 2007). Pursuing this mission, the system of continuous training is regulated by diverse legal diplomas. We will now present their central dimensions. Continuous training first appears in 1986 in the Basic Law of the Educational System (Lei de bases do Sistema Educativo)1. Subsequently, in 1989, a Law2 recognizes the importance of continuous training in the scientific and pedagogical domains of the educators' competencies and asserts continuous training as a condition for career progression. Also the Statutes for the Careers in Teaching at the Elementary and 3 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- Secondary Levels (Estatuto da Carreira dos Professores do Ensino Básico e Secundário), approved in 19903, asserts continuous training as one of the forms of training reproducing what had already been established in 1989 concerning the training initiatives and their intended objectives. In 1992 the Legal Regimen for the Continuous Training of Teachers and Educators (RJFC - Regime Jurídico da Formação Contínua de Professores e Educadores)4 is approved emphasizing the improvement of teaching quality as one of the ends of continuous training. In this document, higher education institutions are seen as having a special vocation to conduct continuous training. But other entities are also authorized to conduct training activities, among them the Training Centres of the Associations of Schools (CFAE - Centros de Formação das Associações de Escolas) resulting from the grouping of schools within a same geographical area that was defined by the Ministry of Education. This legal document defines the following training areas: Sciences of Education and Sciences of Teaching/Learning; Pedagogical Research Practice; Social, Cultural, Personal Training and in Deontology; Portuguese Language and Culture; Information Technologies and Techniques. It also defines the particular modalities the training can assume: Training Courses; Modules; Enrolling in courses from higher education degrees; Seminars; Workshops; Internships; Projects and Circles of studies. CFAEs are governed by the following objectives: to contribute to the promotion of continuous education; to stimulate the exchange and diversity of pedagogical experiences; to promote training needs assessments; to make sure that the training offers are adequate regarding the observed demand for training. It is their responsibility to: secure national priorities in terms of training; establish local priorities; elaborate training plans setting up, if necessary, cooperation protocols with other training entities; broaden their training activities; and to create and manage resource centres. In 19945, the law of 1992 changed and, replacing the former Coordination Council for the Continuous Training of Teachers, a new organ was created, the ScientificalPedagogical Council for Continuous Training (CCPFC - Conselho Científico-Pedagógico da Formação Contínua), and became responsible for the accrediting of training actions. This new organ is more independent from the central administration and has better operational capabilities but also has more fragile connections to the training entities. 4 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- A new law, in 19966, introduced changes in the way continuous training of teachers is regulated, namely by redefining the composition and attributions of the CCPFC. In this Law a new training area is also created: the one regarding the Sciences that are part of the curricula at various levels of education. At the same time, other training entities are recognized, the Training Centres from the Teachers Associations (Centros de Formação das Associações de Professores) and the Training Centres from the not-for-profit Professional or Scientific Associations (Centros de Formação das Associações Profissionais ou Científicas sem fins lucrativos). New objectives are stated, reinforcing the self-training of teachers and educational innovation practices, in order to respond to the training needs identified and expressed by the educational institutions and their teachers and educators. New competencies (areas of authority) are formulated: to coordinate and support innovation projects from educational institutions and to promote the articulation of projects developed by schools with the local administration organs, therefore striving to involve communities and to give an impulse to school autonomy. In 20057, new changes were introduced. They establish that, for career progression purposes, at least 50% of all the training credits must be from training completed in the teaching areas of the teacher. This tendency is reinforced by the new Statute for the Teaching Career8, that increases to a minimum of two thirds the amount of training actions in the teacher's scientific-pedagogical area. Along this period, several studies were conducted on the teachers' continuous training system9 mainly regarding the activities of the CFAE, the training entities in closer proximity to the schools, and their relation to the institutional work of schools and the professional practices of teachers. From a first approach to the results and conclusions of those studies it is possible to identify the main strengths and vulnerabilities of the system. By institutionalizing the constitution of CFAEs and by defining their organization in a way that implicated school organization, the implementation of the system of continuous training of teachers professed its articulation with educational territorialization and the possibility of construction of strong training dynamics centred around the reality of schools and their educational problems, while bringing incentives to the formation of school associations. However, that intention came to be subverted by the inner logic of the 5 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- development of the system. For one, by connecting continuous training of teachers to their career progression its value of ‘exchange’ was emphasized in opposition to its value of 'use', leading to a tendency to choose the training areas and modalities that are more in accordance with a market logic (the greater number of credits being exchanged for the smaller personal investment) and not necessarily the ones that are more conducive to professional and institutional practices. On the other end, the continuous training system's dependency on the funding system came to condition the whole process of accreditation and development of training actions, bringing nefarious consequences to its implication in the conception of individual and school-wide training plans. The conceptualization and realization of these plans, even though it is foreseen in the way the continuous training of teachers is regulated as well as in the diploma on the management and autonomy of schools, ended up being one of the system's main failures and so one of the aspects upon which it is urgent to resume debate and reflection. CFAEs' activity became the captive of state-defined priorities and the funding policies attached to them, and of teachers' career progression logic, as well of the multiple individual and corporate logics pertaining to the universe of trainers. Since the system has been developing according to logics and reasoning that do not include these dimensions, it is then not strange that few self-evaluation studies can be found addressing the impact of continuous training on the quality of the work done by schools and on the education of children. However, after 1996, a time in which the system acquires new stability and coherence, CCPFC initiated an organized set of actions with some training entities (namely the CFAEs) aimed at enhancing the connection between continuous training efforts and school practices, i.e., aimed at overcoming one of the main criticisms directed at the continuous training system: the absence of a transforming impact on educational practices. Data appears to indicate that this set of actions was effective. In fact, training actions focused on school contexts increased from 11%, in 1997, to 63%, in 2006 (82 in total, 7 being from CFAEs). Meanwhile, the qualification of the trainers involved also improved - at that time 13 300 of the trainers had completed postgraduate training. In spite of several deficiencies - and we underline the lack of relevance in terms of the transformation of practices - there are also several positive aspects in the system that 6 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- can be pointed out: the creation of a culture of training and an attitude of innovation and of responding to educational policy challenges, namely in the curriculum domain, the improvement of the conceptual quality of the accredited actions, an increased awareness regarding the need to improve performance, and the advances in the idea of articulated and coherent training plans based on individual and school-wide plans. In fact, several studies tell us of institutional initiatives and training projects that counteract the negative tendencies and find their place at the margins of the logics that were taking over the system. These projects, emerging in educational contexts, proposed by CFAEs or developed in partnerships combining educational and research institutions, had highly significant impacts on teaching practices, on the education of children and youth, and on the organization of schools and the levels of quality of life within them. Options on the field of study and methodological procedures From the partial results on this axis in the assessment process and from small interviews conducted with key informants some fundamental choices regarding the research design were made: the training periods and entities to be considered, and the global strategy for gathering opinions using questionnaires and collecting information on the training entities. In 2006 there were 391 training entities and 14 200 accredited actions. From the training entities, 52% were CFAEs and 73,5% of the accredited training actions were from the CFAEs. Meanwhile, CFAEs offer most of the amount of training that has the biggest potential to impact school and teacher practices. Considering this, and regarding the main focus of the assessment, the training entities that will be studied will be only the CFAEs. The choice of periods (civil years) to study took into account that: the system of evaluation acquired its present form after 1996; that in the 10 following years, apparently, training came progressively closer to the 'territory' (namely through training offers and modalities that are closer to practice, and as a consequence of the 'pedagogical relation' CCPFC assumed with the CFAE); that the stability of the training offer is reduced at the beginning and end of each funding program; that legal changes regarding the obligatory 7 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- character of training in the teaching area and educational level (in 2005); that alterations made only came to mean something some time after they are in effect; and that, after the end of the 20th century, the system tried to respond, in a clearer manner, to the challenges posed by educational policies - those pertaining to school autonomy and the managing of curricula - and to the demands of knowledge society. So, we elected three periods: 1998, 2002 and 2006. We were still facing 192 CFAEs stretching all along the national territory and thousands of teachers, trainers and schools. This raised issues in terms of the possibilities of real research, and without sustainable knowledge regarding the adequacy of the instruments for more extensive data collection: self-assessment protocols from training entities and questionnaire surveys. Progressively, it became clear that the success of the assessment project would come from the establishment of a collaborative relation between the research team and the training entities being assessed. Since the CFAEs are organized according to regions, 5 regions, each one of them being coordinated by one of the directors of one the CFAEs in the region, and that each region is structured in networks (26 in total), that agreement would be made between the regional coordinators of the CFAEs, the research team and the Ministry of Education represented by the CCPFC. In this agreement, the sampling procedures, and the ways of communicating - distribution and return - the information among the instances could be clarified and the principles guiding the data collection could be made explicit. Meanwhile the assessment process continues with interviews and case studies that fulfill their own objectives and support a grounded and adequate construction of data collection instruments, both quantitative and other (broader) ones. These same means and modes of research will be employed after the data treatment in order to deepen the interpretation of some of the results. Given its key importance to the research process, we will now underscore some of the basic decisions made in terms of the self-assessment protocols for the CFAEs and the gathering of the opinions of teachers. 8 15ème congrès AMSE AMCE WAER, Marrakech, 2-6 juin 2008 ----------------------------------------------------------------------------------------------------------- Self-assessment of the Training Centres from School Associations (CFAEs - Centros de Formação de Associação de Escolas): some basic decisions Previous studies on the assessment of the impact of training, the ones we have already mentioned as well as others10 focused on the role of the CFAEs, have not been able to provide a deep understanding of the way they accomplish their objectives or of the internal functioning of these centres both in general terms and in terms of each of the particular centres. Actually, even though it seems to be the general opinion that the CFAEs - respecting the pedagogical autonomy they possess (Article 21.st, RJFC) and the bounds of a common normative framework - have been instituting their own dynamics and constructing their own path, and in doing so becoming more and more distinct from each other, this idea does not find solid support in the empirical data. Collecting data through self-assessment procedures is justified by the recognition of the need for a deep knowledge of the way CFAEs work. As it has already been said, there are not many self-assessment studies of the impact of training on the work done by schools, and also, not many studies have emphasized processes of self-assessment of the activity and functioning of CFAEs, an activity that obviously has repercussions for the ways they communicate with the schools and the manner in which training plans are organized and managed. In the framework for the assessment (CCPFC, 1999), one can find aspects concerning the role of CFAEs in organizing and managing training, specifically in terms of: i) the options regarding training modalities; ii) the criteria that guide the organization of training actions, namely the procedures used to identify training needs (CCPFC, 1999: 102103); iii) the way in which CFAEs welcomed once-off training initiatives as well as other initiatives that are originated in the schools. Other aspects mentioned intend to investigate: 1) the manner in which CFAEs see their part in continuous training and the degree to which they understand its underlying structure of objectives; 2) the way CFAEs articulate initial and continuous training actions; 3) the degree of involvement in the organization of future specialized training courses that they can foresee. 9