Modeling authentic multivariate assessment in art and design teacher education curriculum

Modeling authentic multivariate assessment in art and design teacher education curriculum Modelización de la evaluación multivariante auténtica en el plan de estudios de formación de profesores de arte y diseño | Modelagem de avaliação multivariada autêntica no currículo de formação de professores de arte e design ATTWELL MAMVUTO · [email protected] UNIVERSITY OF ZIMBABWE · ZIMBABWE http://orcid.org/0000-0003-2644-338X Recibido · Recebido · Received: 07/11/2023 | Aceptado · Aceito · Accepted: 11/12/2023 DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 Artículo bajo licencia Creative Commons BY-NC-SA · Artigo sob licença Creative Commons BYNC-SA · Article under Creative Commons license BY-NC-SA. Cómo citar este artículo · Como citar este artigo · How to cite this article: Mamvuto, A. (2023). Modeling authentic multivariate assessment in art and design teacher education curriculum, Communiars. Revista de Imagen, Artes y Educación Crítica y Social, 10, 24-36. https://dx.doi.org/10.12795/Communiars.2023.i10.02 Abstract: Assessment is a critical construct in programing and instruction. It is one of the means of accountability that is conceived within the broader context of authentic performance. This paper discusses assessment strategies that are being used in art and design teacher education programmes amid ongoing curricula reforms in teachers' colleges in Zimbabwe. Findings reveal that colleges are using diverse assessment strategies to index student performance. These include critique of student exhibitions, portfolio assessment, pre-issue examinations, and controlled examinations, all of which are operationalised within an internal and external moderation framework and dialogical engagement among stakeholders. The study proposes the inclusion of candidate interviews and deliberate student self-assessment as part of a comprehensive art programme to enhance the development of higher-order cognitive and metacognitive skills that are critical in enhancing aesthetic judgment and performative art. Keywords: Art curriculum. Art education. Assessment in art. Authentic assessment. Portfolio assessment. Teacher education. Resumen: La evaluación es un constructo fundamental en la programación y la instrucción. Es uno de los medios de rendición de cuentas que se concibe dentro del contexto más amplio del rendimiento auténtico. En este artículo se analizan las estrategias de evaluación que se utilizan en los programas de formación de profesores de arte y diseño en el marco de las reformas curriculares que se están Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 25 llevando a cabo en las escuelas regulares de Zimbabue. Los resultados revelan que las escuelas utilizan diversas estrategias de evaluación para valorar el rendimiento de los estudiantes. Entre ellas se encuentran la crítica de las exposiciones de los estudiantes, la evaluación de las carpetas de trabajo, los exámenes previos a la expedición y los exámenes controlados, todo ello dentro de un marco de moderación interna y externa y de diálogo entre las partes interesadas. El estudio propone la inclusión de entrevistas con los candidatos y la autoevaluación deliberada de los estudiantes como parte de un programa artístico integral para potenciar el desarrollo de habilidades cognitivas y metacognitivas de orden superior que son fundamentales para mejorar el juicio estético y el arte performativo. Palabras clave: Plan de estudios de arte. Educación artística. Evaluación en arte. Evaluación auténtica. Evaluación de portafolios. Formación de profesores. Resumo: A avaliação é uma construção fundamental na programação e na instrução. É um dos meios de prestação de contas concebido dentro do contexto mais amplo do desempenho autêntico. Este artigo discute as estratégias de avaliação que estão sendo usadas nos programas de formação de professores de arte e design em meio às reformas curriculares em andamento nas faculdades de professores no Zimbábue. Os resultados revelam que as faculdades estão usando diversas estratégias de avaliação para indexar o desempenho dos alunos. Essas estratégias incluem a crítica das exposições dos alunos, avaliação de portfólio, exames pré-edital e exames controlados, todos operacionalizados dentro de uma estrutura de moderação interna e externa e engajamento dialógico entre as partes interessadas. O estudo propõe a inclusão de entrevistas com candidatos e a autoavaliação deliberada dos alunos como parte de um programa de arte abrangente para aprimorar o desenvolvimento de habilidades cognitivas e metacognitivas de ordem superior, que são fundamentais para melhorar o julgamento estético e a arte performática. Palavras-chave: Currículo de arte. Educação artística. Avaliação em arte. Avaliação autêntica. Avaliação de portfolio. Formação de professores. … 1. Introduction Assessment transcends educational endeavours and remains the hallmark quality index in determining performance. It is equally important to teacher education in the management of activities embedded within art programmes. In many teacher education programmes assessment has always been conducted both internally and externally. In Zimbabwe teachers' colleges, the implementation of internal and external assessment has remained conceptually unaltered since its adoption in the 1950s under the supervision and monitoring of associate institutions by a University. Now there are emerging assessment modalities that are being used by colleges in an attempt to integrate more effective internal assessment strategies for art and design like in other curricula disciplines. These strategies are applied to both the pedagogical curriculum and the main study self-enrichment syllabus. In Zimbabwe teachers’ colleges, students are externally examined at the exit stage of their education programme. The current practice in the external assessment of students is such that the grade awarded by an external moderator takes precedence over the internal gradea position that has stirred contestation and debate among internal examiners. Internal examiners have expressed a preference for more credit being accorded to their contribution since students spend more time under their supervision and guidance. Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 26 However, the final grades are deliberated on in a dialogical way as provided for in the assessment regulations (DTE Handbook, 2015; SSAAS Handbook, 2011). The grades are jointly ratified by a board of internal and external assessors under the supervision and guidance of an academic chief examiner appointed to the associate college by the accrediting university to oversee the authenticity and fairness of the examination process. The results are further ratified by the legally constituted Board of Examiners and Board of Studies before endorsement by the Senate as the final certificating authority of the University. In the current arrangement, external moderation in colleges is done in the final year of study irrespective of the duration of the programme. Colleges offer two, three, or oneand-a-half-year programmes and in some cases two or three of the programmes. For a three-year programme, for example, third-year internal grades are the ones mostly considered for the final assessment of the candidate’s performance. The assumption is that students would have matured and gained skills, knowledge, and competencies commensurate with the exit stage. However, there seems to exist a missing dimension in profiling student performance across the duration of the programme. For instance, to what extent does a student’s learning during the programme contribute toward the final assessment? Assessment at various levels, therefore, becomes a grey area that prompted this study. These curricula gaps point to a need to rethink assessment for enhanced efficiency and effectiveness and its alignment to different educational performance levels. This paper, therefore, sought to interrogate the internal and external assessment processes being used in art and design diploma in education (teacher education) programmes to improve the efficacy of the assessment practice. The following questions guided the study: 1. To what extent do the assessment methods in practice in teachers' college art programmes authenticate the artistic and professional performance and growth of students? 2. How effective are these internal and external procedures in assessing artistic performance? 3. What is the level of dialogical engagement between and among internal and external examiners in the assessment of student artistic performance? 4. To what extent is student engagement an integral part of the assessment matrix? 2. Assessment in the visual arts According to Clark (2002), assessment in expressive arts has always been a contested terrain because of its challenges in addressing complex issues of objectivity. Over the years, the arts have consistently been perceived as non-academic disciplines leading to their general neglect and exclusion from the assessment regime. However, they emerged in the mainstream assessment regiment coming from the mid-20th century accountability movement, when subjects had to justify their place and role in the school curriculum despite objectivity challenges. This now calls for creative ways of assessing learners using methodologies that are outside the traditional realm of standardised criteria because of the expressive nature of the arts. In higher education, there is the emergence of a ‘…proliferation of more diverse and experimental forms of assessment, many utilizing digital media and multimodal expression, which are aimed at facilitating and evidencing multiple literacies and skills sets but in which assessment criteria are Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 27 contested and often highly subjective’ (Walker & Gleaves 2008, p. 41). Similarly, Kleiman (2005) observes that there has been a movement towards the use of authentic assessments (Heijnen, Braam & van Tongeren 2021) in line with constructivist views about pedagogy. Walker and Gleaves (2008) further observe that while assessing creativity is highly subjective, however, that does not warrant the use of rigid versions of assessment as intrinsically more desirable. They advocate for narratives that facilitate the understanding of art concepts and skills. Guskey (2001) argues that such tracking of student performance reveals their learning needs which call for multiple assessment strategies. Assessment standards generally fall under three educational domains: knowledge (cognitive), skills (metacognitive), and values (affective) (Human & van Niekerk 2014), which are visible in the arts, ‘[i]n African musical arts, life, and arts are integrated as an inter-art and inter-life experience’ (p.27). Thus ‘[t]he assessment of intangible and unobservable thought or enculturation processes, which form an integral part of arts education, is made possible because the process of thought, as well as the evaluation of thought process, can be verbalized, analyzed and structured into a process description with levels’ (p.29). Therefore, teaching, learning, and assessment inform the structuring of performance standards framework for the arts. As one acquires knowledge, skills, and values in the arts, one goes through the novice, proficient, and master performance levels. A survey by Burton (2001) came up with several qualitative means and forms of assessment in the arts which include direct observation, behavior and attitude assessment, formal grading, and the use of portfolios. With regard to portfolios, Clark (2012: 31) notes that ‘[m]any teacher education programs in colleges and universities have a portfolio requirement for graduation…’ Portfolios document growth over time across a range of activities (Barton & Collins, 1997; Mamvuto & Kangai, 2021; Paulson & Paulson, 1991; Saunders et al, 1999). Elton (2006) advocates for the use of portfolios as ‘interpretive assessment methods’ for problem-solving. Similarly, Balchin (2005) discusses the complexity of assessing creativity and the anxiety such endeavours put on students. In addition to the aforementioned, the other assessment modes include critiques which can cause students to reflect on their work and hypothesise strategies for improvement, demonstrations, and working exhibitions. Video clips of artifacts, annotated sketchbooks, instructional games, written research papers, presentations, class discussions, tests, quizzes, and examinations, have historically been used to bring diversity and objectivity to the assessment process. Indicators of student learning demand considering both the process and product approaches to assessment. While ‘[p]roduct refers to the aesthetic quality of the student’s finished result or work of art’ on the other hand process refers to ‘the entire range of activities required to produce a work of visual or performance art, from concept to creation’ (Clark, 2002, p. 30). He further recommends for a balance between product and process approaches arguing that the former does not necessarily reveal the level of understanding, depth of skills, commitment effort, creative thought and personal meaning of the process that creates the work. In that regard, Eisner (1985) cited in Clark (2012, p.30) argues that the focus should be on the ‘cultivation of intellectual power’ hence the importance of a process-based approaches to assessment. Articulating the same argument, Peeno in Clark (2012, p.30) intimates that ‘documenting what students know and can do is essential, but to continue to report this information with only a number or letter grade, however convenient, is insufficient and not helpful as an indicator of progress in art.’ Thus Guskey’s (2001) formative progress criteria are ‘a Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 28 means of assessment [that] typically consider the amount of growth over time rather than focus exclusively on a student’s immediate achievement level.’ Internally assessed assignment tasks, which are a common feature in teacher education in Zimbabwe need to be executed in such a way that they bring some objectivity and precision to the assessment process. Ormond (2011) observes that ‘[i]nternal assessment has been part of a student's grade…but the shift to standards assessment has placed more nationally consistent requirements in the assessment’ (p.578). Thus there is a shift towards ‘…a more student-centered pedagogical approach, placing the student in a more active role in the learning, teaching and assessment cycle’ (Rawlins et al 2010 cited in Ormond, 2011, p.579). Assessment is thus focused on understanding mental processes that give opportunities to practice own analysis. This is an integrated approach to learning that takes into cognisant Bloom’s taxonomy; progression across cognitive levels aimed at developing critical thinking. There is a need to plan for more formative assessment opportunities for both internally assessed and externally examined components, and as such, Elton (2006) and Kleinman (2005) suggest teaching for and about creativity and designing assessment strategies to promote and evidence the creative processes. This is reflected in Heijnen, Braam, and van Tongeren’s model, Figure 1 as cited in Heijnen (2015). The model emphasises authentic art education by interrelating students' culture, learning activities, and context. Figure 1: Authentic art education. Adopted from Heijnen, Braam & van Tongeren cited in Heijnen (2015). 3. Methodology This was a twofold case study of the University of Zimbabwe Scheme of Association (UZSoA) which sought to establish how lecturers implemented assessment strategies in art and design. Inspired by Heijnen, Braam, and van Tongeren’s (2021) authentic model of holistic learning in the arts including assessment, the study involved an analysis of Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 29 existing internal and external assessment procedures and their practice in teachers’ colleges as documented in the DTE Handbook (2015) and college policy frameworks. Data were also generated through analysis of assessment documents from selected teachers’ colleges, which include mark profiles, and syllabuses. The author has participated in the review of examination procedures and processes in the UZSoA. He has also been a University coordinator to different teachers’ colleges and is thus informed about the practice of assessment in art and design in these colleges. Data generation also included answering six broad online interview questions by selected Heads of Art and Design departments. Each administrator has many students at different educational levels under their custody and, are therefore critical participants in college programmes. The participants who were given pseudonyms for anonymity were purposively selected from a total of 18 teachers' colleges. From the set of colleges, I conveniently selected six; two secondary colleges (SC) and four primary colleges (PC). The questions administered through e-mail solicited information on the effectiveness of assessment processes, and methods used to profile student learning in art and design; the extent to which students are engaged in the assessment processes; the effectiveness of the current strategies in assessing artistic knowledge and competencies of students; and suggestions to further enhance internal and external assessment of students in art and design. The Heads of Subject are ideal and key participants in that they chair subject meetings and are the internal assessors in the colleges; they attend College Academic Board meetings as they present candidates for internal examinations. They are familiar with the assessment procedures in the UZSoA; as the custodians of art programmes responsible for the review of curricula, a process which is done after every three years as recommended in the UZSoA (Regulatory Framework 2011). In the next section, I present the results and discussion. 4. Results and discussion These are presented under biographical data and four emerging analytical themes as illustrated in Table 1. Table 1: Participants’ biographical data NAME Kwindas Hungwe Pamela Jedza Mkanya Artnet Attribute/College PC PC PC SC PC SC Sex F M F M M M Age 45 53 47 48 53 49 Professional qualification PhD A&D Med A&D MEd A&D BTE (CAD) MEd A&D MTech A&D Year Obtained 2021 2017 2018 2006 2017 2018 Teaching art at college (years) 17 15 14 15 12 17 Administrative position HOD PL HOS LIC HOS HOD Key: HODHead of Department, PLPrincipal Lecturer, HOSHead of Subject, LICLecturer in Charge. Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 30 The participants (2 females and 4 males) were aged between 45 and 53 years. Of the 6 participants, 1 had a Ph.D., 4 had a master's degree and one had a bachelor of technology education degree (creative art and design – CAD) respectively. By any standards, these are appropriate qualifications to teach at the diploma level. Two were teaching at a secondary teachers' college while the other 4 were teaching at a primary teachers' college. All had vast experience (between 12 and 17 years) teaching art at the college level. Similarly, all participants were senior lecturers (SL) who held administrative positions in the college making them qualified to give innovative suggestions on how to assess and implement new strategies of assessment in the UZSoA. 5. Internal processes, methods, and tools being used to assess students in art and design In the UZSoA several processes and methods are being used to assess student performance. These include written assignments, artifacts in portfolios, digital portfolios, exhibitions, tests, oral presentations, and rubrics. These lead to examinations, both theory and practical. According to a female participant, Pamela, Students are assessed through a variety of tasks which include both practical and theory assignments, mini-research, and community projects. A variety of processes are used such as observations as they carry out practical work to pick on areas for improvement in skills development, and critique sessions to establish their ability to judge works of art. They also carry out presentations to allow fellow students to input and the lecturer to also question and assess their grasp of concepts. Both formative and summative assessment of theory and practical assignments which are then graded before the end of programme examinations are given to students. Thus the lecturers use continuous assessment processes to make students creative, analytic, and to enable them to judge their work. Self and class assessments in the critique of artworks are also used. Hungwe emphasises that, Students are also given tasks in the form of portfolios which they are compelled to do during their teaching practice period. Continuous supervision of these portfolio tasks is done during and after their teaching practice experience. Discussions are done with students when they carry out their assignments and tasks both during their stint in college and when they are on teaching practice. This is done to give students direction in executing tasks. During the final third-year session, students are assessed in foundational areas where they are exposed to drawing, painting, graphic design, textiles, and construction among other components. They then each choose one specialisation area and are then assigned to a lecturer for supervision and tutoring. Assessment tools that are used include a combination of portfolios (digital portfolio included), marking guides, observation guides, and tests and exercises that are used to score and assess students’ performance levels. Design processes folios, write-ups, assessment crits, mark profiles, rubrics, and concept paper presentations are also used for both theory and practical assignments. Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 31 6. Strategies for assessing artistic knowledge and competencies As a strategy to ascertain objectivity in the assessment of student performance, participants indicated that they have integrated, team marking for practical tasks guided by the marking schemes we will have designed collectively. The theory assignments are shared among members of the department. Through team marking of practical tasks, biases are minimised to a greater extent since there is thorough scrutiny and some standardisation from all members of the department. Marking guides also help in the standardisation of the mark given to a candidate. Portfolios allow students to develop their artistic skills in the four areas they do; drawing, painting, graphic design, textiles, and construction/3D. Assessment is solely by art lecturers and students are not involved when they should be allowed to assess themselves. In that regard, the assessment strategies may not be fully beneficial as they leave out the student who is an important stakeholder in the assessment process. (Hungwe) This strategy is viewed as effective especially from a continuous assessment perspective where students start from design briefs through to the engagement of actual artistic activities whereby there is interaction between the student and the lecturer. Pamela similarly indicated that, Presentations have proven to be effective as students can research widely and develop their confidence, particularly with theory topics such as Western art movements and Zimbabwean art, which have wide content that cannot be covered by the lecturer since time is limited. Students retain content better when they research and present themselves as opposed to being fed during lectures. Discussions of artworks during critique sessions have proven to develop students' analytic skills. Students are given coursework assignments, both theory and practical. However, they tend to do better in practical assessments some of which they do off-campus. One disadvantage of doing practical work off-campus is that students can assign other people, such as professional artists to do the work for them. However, in the final assessments, students do both theory and practical examinations under examination conditions. Participants indicated that overall, the assessment strategies are working well as shown by the quality of work produced by students that seem to show upward progression. According to Artnet, ‘The strategies are quite effective and they meet the expectations. However, there is need for an external assessor to moderate students’ practical work. This used to be the case but it was called off.’ Kwindas however indicated that the participation of students in self-assessment is very minimal while Mukanya observed that at their college ‘Students are given the opportunity for self-assessment and also to critique each other’s work.’ On the contrary, Kwindas averred that ‘there is minimal involvement of students through oral presentations and class discussions’ as similarly observed by Hungwe, ‘Unfortunately students are not directly involved in the assessment process. Their role is to produce work as assigned to them by the department.’ Similarly, Jedza indicated that ‘[students] are involved from the onset up to the end of the production of the work, consulting peers, professionals, and lecturers before a summative assessment is done.’ According to Pamela, Students have to acquire materials needed for practical activities and they have to produce artwork within the stipulated time as the lecturer has to ascertain that the work done is authentic. The lecturer may demonstrate processes or use YouTube tutorials. Students work both individually and in groups during the assessment. Mamvuto, 2023 · Communiars, 10, 24-36 COMMUNIARS · ISSN 2603-6681 · DOI: https://dx.doi.org/10.12795/Communiars.2023.i10.02 · 32 Where corrections or emphasis needs to be made the lecturer may interrupt the session to raise such issues. Students also take note of the key issues in their notebooks during such sessions. 7. Enhancing internal and external assessment systems in art and design in the UZSoA Participants suggested strategies that can be implemented at both internal and external levels to improve performance in the UZsoA. As Hungwe suggested, Students should be allowed to do an objective assessment of their art productions where the individual student or peer assigns a mark to his/her work. I also feel that members of staff in the college who have some related knowledge of art production and criticism such as Textiles staff of the Home Economics department should also be allowed to assess some artworks in our department to enhance quality, seriousness, and objectivity in the production of art and design work. It was suggested that internally there is a need to engage students in portfolio assessment by assessing the process and handling of materials as well as the final product. The Centre for Teacher Education and Materials Development (CTEMD) as the custodian of the diploma programme should also outline its expectations for the authenticity of the internal systems. Furthermore, Artnet suggested that the CTEMD at some point should organize workshops where parameters and minimum expectations are set with the involvement of all stakeholders. Pamela further suggested that, Whilst the theory is important in art, it is the practical component that brings out the real art student. As such, weighting of the practical to theory should be higher, for example, 80:20 respectively so that students who have the aptitude for art are identified and rewarded. There is a need to treat practical subjects differently from theorybased disciplines such as the Theory of Education. Jedza similarly suggested considering the weighting given to coursework, which he currently views as being too low. He suggested giving coursework 60% against 40% examination as opposed to the current 70% examination. He further suggested that, ‘Students can carry out art-related community projects in schools to improve perceptions and attitudes towards the subject. This will ensure that more children appreciate the subject and take it up to tertiary level.’ Mkanya suggested implementing the researchguided assessment. He however notes that literature is scarce on the effectiveness of assessment methods used at a local level. Hungwe pointed out that, External assessment should not be summative. There should be continuous assessment of art by the CTEMD from the first year to the final third year. This is important because it would enable meaningful growth within the art education fraternity. Assessment of art should not be for fulfillment or completion of a course of study but to enhance skills for national development, especially in line with the new philosophy of Education 5.0 Furthermore, real collaboration and interaction between the college art department and the CTEMD on ways of improving and managing assessment processes should be fostered at reasonable intervals of time during the year, if it is possible timelines can be set. The available literature on assessment in the UZSoA indicates displeasure among lecturers on the advantageous position given to external assessors whose grades they say override those of internal assessors. As Pamela observed,