Towards greater objectivity in classroom observation

Towards greater objectivity in classroom observation Graham Howells Universidade de Brasilia Lynne Meiring University of Wales Many teacher trainers have struggled with classroom observation schedules that have tended to consist of grids in which the frequencies of a specified range of teacherlearner interactions are marked. These schedules were originally planned before the miniaturisation of video cameras had made it practical to film lessons relatively unobtrusively. This article argües in favour of a system in which, when classes are filmed, the results may be analysed in conjunction with grid-based evaluation schedules and presented via computer-generated graphs to créate an objective and easily interpreted method of reporting to the student teacher the distribution of classroom activities. These activities may then become the subject of discussion with reference to their instructional valué. 1. The Increasing Importance of Classroom Observation The increase in inspection and appraisal in education today means that there is a general need for teachers to have a guarantee of receiving the most objective means of appraisal possible and also to be given a means of preparing themselves for appraisal. Much work has been done in devising observation schedules that describe and analyse classroom discourse and interaction, but there are problems in using such procedures in order to evalúate teacher effectiveness: I. They have often been devised and are often carried out by an observer who does not regularly teach (and indeed may not have been a classroom teacher for a considerable time). The process may become a "top-down" exercise rather than a sympathetic, collaborative one. (Also, it is very rare for learner feedback to be taken into account concerning the lessons observed, unless the learners are adults.) II. The schedules have usually been devised according to a tallying system on a printed grid. This method gives a quantitative summation of interactions from which qualitative conclusions must be inferred after analysing total numbers of interactions. Some of these systems are described below. Revista de Lenguas para Fines Específicos N°-" 5 y 6 (1998-1999) 241 Graham Howells and Lynne Meiring III. The observación techniques are problematical in that a schedule of easily manageable proportions gives only a partial view of classroom interactions. On the other hand, it is difficult in nonexperimental situations to opérate systems that pre-specify a wider range of teaching/learning activities to be noted. IV. There seems to be an assumption (either explicit or implicit) behind research in this área that the analysis of classroom interaction allows teacher trainers to enable students to conform to some ideal teaching style when, as the Pennsylvania Project (Clark, 1969) indicates, we cannot even state with certainty what overall teaching approach is the most effective. V. Observación schedules have often been developed through research in applied linguistics and in teacher training and appraisal, have been made to perform a task that they were not designed for. They do not take into account variables such as: socio-economic environment of the school; learner motivation; quality of teaching materials; the physical environment of the classroom, or the range and quality of its technical equipment. VI. From a technical point of view there is a problem in that some of the appraisal methods described in the literature were formulated before the widespread use of video-recording. Where videorecording has been used, it has usually merely provided a more methodical way of describing interaction, without always taking sufficient advantage of the opportunities offered for analysis of the teaching activities. VIL Observation schedules have a centralising tendency: teacher/ learner behaviours have to fit the Procrustean bed of the headings on the report form. Those categories are used to assess the teacher's performance even though, as suggested above, it is impractical to créate a usable report form that will encompass all the behaviour variables of normal classroom interaction. It is not therefore surprising that these observation systems pay little heed to variables of individual teaching styles. 242 Revista de Lenguas para Fines Específicos N°" 5 y 6 (1998 y 1999) Toivards greater objectivity in classroom observation 2. The Desiderata of Classroom Observation What is needed is a means of achieving a system of observation that is seen to be fair and objective, easy to implement and which offers the possibility of clear presentation of results to the teacher being observed. Firstly the wliole área of classroom observation must be considered in its methodology and its effectiveness. The recent move towards accountability and teacher appraisal will no doubt have highlighted the need to carry out accurate classroom observation. This observation must look beyond the teacher to what is actually taking place in the classroom. Ellis (1990:68) quotes Mclaughlin's view on this issue claiming that there are dangers in viewing teacher/learner behaviour separately because Information is lost about the sequential flow of classroom activities. There are clearly problems with this in that observers have the formidable task of evaluating the teacher, the pupils and their interaction. This assessment must be fair and provide a basis for discussion and therefore must be carefully backed up by fact. It must be far more accurate and beneficial than a "mock bureaucratic ritual", and it must involve "performance review, development review and potential review" for pupils and teacher. Concentrating on any individual focus would fail to appreciate the synergistic wholeness of the lesson. In addition to this there is the problem of subjectivity. Often during concurrent appraisal the teacher is provided with a slightly biased unsubstantiated appraisal. Fanselow (1977) pinpoints in a rather colourful way the problems involved in appraising lessons thus: When teachers, employers, students or sales people discuss the same lessons, texts, tests, methods and schools of language teaching they often sound like the characters in the Japanese movie, Rashomon - they each give contradictory and equivoca! accounts of the same events or Ítems. He then goes on to describe an instrument - FOCUS - for observing communication used in specific settings. Revista de Lenguas para Fines Específicos N""^ 5 y 6 (1998 y 1999) 243 Graham Howells and Lynne Meiring The simple conclusión is that meaningful evaluations cannot be made unless an accurate picture can be given of what is actually going on in lessons. A more accurate picture of lessons can be gained by using an observation instrument to créate a picture of interaction, which gives a concrete picture of verbal behaviour. Such systems opérate on a series of categories which the observer tallies whenever the particular category is discernible in the lesson. These can then be entered onto a matrix to provide a graphic picture of the lesson. This enables teaching/learning patterns to be analysed. Percentages for each category can be calculated and ratios discovered. Most instruments of lesson analysis are essentially adaptations of the original system developed by Flanders. Instruments should include an analysis of teacher talk and of pupil talk and also some way of monitoring other types of verbal behaviour and the absence of it. Although in themselves they do not demónstrate how meaningful the interaction is, they do provide useful data for this purpose. Many such instruments have been created and trialled since the 1960s, some specifically concerned with language acquisition and others more general and appropriate to a greater or lesser degree to various curriculum áreas. These instruments have differing numbers of categories which are ticked as appropriate at time intervals during a lesson, for example every 3 seconds. Appendix 1 shows some language-specific models quoted by Chaudron (1988), based on research by Long (1976). In this selection the number of categories varíes from seven to 73 and there is also variety in the specific time period involved for observation. It would be impractical within the confines of this article to attempt to analyse the many instruments that have been developed to date; it is sufficient to highlight some aspects of them and mention some of the more famous instruments. Perhaps the most well known and certainly the original is the Flanders system, described by Medley and Mitzel (1963) as «the most sophisticated technique for observing classroom climate» [quoted in Allwright (1988:68)]. 244 Revista de Lenguas para Fines Específicos N"'' 5 y 6 (1998 y 1999) Towards greater ohjecHvity in dassroom observation Flanders employed ten categories, seven were teacher-related and these were divided into direct and indirect ones, then carne two student-related and one silent category. «Teacher talk» included the categories of accepting feelings, praising and encouraging, accepting or using ideas of pupils, asking questions, lecturing, giving directions, criticising or justifying authority. The pupil categories were pupil response and pupil initiation. There was then a silent category to cover pauses or any confused communication which could not be understood by the observer. Moskovitz (1976) created another well-known instrument which is worth mentioning because it took the Flanders categories and, by making additions and adaptations, made an instrument more suitable to the Modern Language Classroom. Under «Pupil talk» were included choral response, reading aloud, responding to the teacher with opinions and feelings, being off the subject, being non-task orientated. The teacher categories were extended to include jokes, repeating student response verbatim, asking cultural questions, discussion, personalising, modelling and several others. Jarvis' (1968) model is also quite impressive because of the way its 24 categories capture characteristics of the Modern Language classroom, differentiating between drill language and language for real communicative purposes. A more recent and perhaps less cumbersome instrument is that of Bowers (1980) which drastically reduces the number of categories to seven. Bowers also includes the vital differentiation between language used directly to teach, and language used for normal social and organisational purposes. There are obviously many problems involved with using these Instruments: firstly, using an existing instrument may not enable the researcher to achieve the focus required for a specific purpose and secondly it can take a very long time to become sufficiently acquainted with an instrument to be able to use it effectively, a point Revista de Lenguas para Fines Específicos N"" 5 y 6 (1998 y 1999) 245 Graham Howells and Lynne Meiring not lost on Moskovitz (1967), who produced a programmed manual for teachers to teach themselves how to use the Flanders' system. 3. The Benefits of Classroom observation. Another issue which needs to be addressed is the usefulness of these instruments in classroom observation. Teacher trainers and those carrying out any form of appraisal would no doubt welcome any system which took the overriding subjectivity from teacher assessment and made it more valid. Dick Allwright gave a paper on this in 1972 entitled «Prescription and Description in the Training of Language Teachers». He outlined the problems faced by supervisors as they carry out prescriptive assessment and arrived at the conclusión that prescription is inappropriate if descriptive techniques are inadequate. A teacher trainer is far better employed encouraging students to examine themselves what actually goes on in lessons. In any discussion from the perspective of the trainers, it is important to be acquainted with the work of Grittner, State Supervisor of Foreign Language with the Wisconsin Department of Public Instruction in 1969. In Teaching Foreign Languages (1969) he concluded, from his own visiting of schools, that teachers have a distorted view of what actually takes place in their lessons, and are astonished when they hear an audio recording of those lessons. Even in what was supposedly an audio lingual lesson, 80% of the classroom work involved teacher talk, most of which was in English, and on the rare occasions when the pupils did speak, their responses were either repetitions of what the teacher had said or responses elicited from written material. All functional communication carried out between teacher and pupils was in English. The teacher's use of the target language was minimal and pupil use nil. In the light of these observations Grittner became interested in what he termed «interaction analysis» as a means of changing teacher behaviour. Subsequently Grittner dropped his role as a supervisor and became involved in encouraging and instigating teacher self observation. He spent 246 Revista de Lenguas para Fines Específicos N°' 5 y 6 (1998 y 1999) Towards greater objectivity in classroom observation a considerable amount of time worlíing on publishing Nearhoof's adaptation of the Flanders system which stressed the need to distinguisli between real and drill use of language. Moskovitz not only saw very clearly the benefit of using observation Systems as a solution to supervisor problems, but also the fundamental benefit in terms of trainee feedback. What she was in fact advocating was a trainee centred approach to training teachers. Teachers can find out many things from the results of this analysis. Moskovitz lists some of these as follows: 1. What percentage of the class time does the teacher talk? 2. What percentage of the class time do the pupils talk? 3. Does the teacher use more direct or indirect influence in the lesson? 4. Is the teacher more direct or indirect in motivating the class? 5. What immediate feedback do pupils receive? 6. To what extent is there pupil participation for extended periods of time? 7. What behaviour is used to elicit pupil response? 8. To what extent can pupils bring in their own ideas? 9. What behaviour does the teacher use extensively in communicating? These observations clearly illustrate something of the valué in terms of personal development. Although evidence of the benefit to teachers is sparse, Moskovitz does refer to some research in this field in her paper, «The effects of training foreign language teachers in interaction analysis». The research referred to was carried out on both in-service and trainee teachers. Fourteen subjects were asked to assess their teaching by using the Flanders system of classroom observation. It was discovered as a result, that teachers used more indirect behaviours in motivating and controlling their classes, more praise, less direct influence and there was more acceptance of pupils' feelings and ideas and less criticism. The students on the other hand, were observed to initiate their own ideas more, give fewer naRevista de Lenguas para Fines Específicos N"'' 5 y 6 (1998 y 1999) 247 Graham Howells and Lynne Meiring rrow predictable replies, initiate the conversation more and talk for greater lengths of time when they expressed their own ideas. At the end of the training time the student teachers were asked for their evaluation of the system. They were asked initially whether the study of interaction analysis applies to foreign language teaching. They were given a scale of 1-9, with 1 indicating 'not at all' and 9 'a great deal'. The mean score for the group was 7.8 indicating that they felt the Flanders' system had considerable application. They were then asked if they felt the study of interaction should be made a requirement for language teachers, six strongly agreed, eight agreed, no-one chose neutral, disagree or strongly disagree. Two other open-ended questions were also asked. In the first instance they had to complete the phrase, «study of interaction analysis has made me realise...» Responses included: ...that I can plan the way I am going to behave in the classroom, what is going on in the classroom and what is successful under specific conditions; exactly what I am doing in front of the class both good and bad, how my students react to my behaviour, that controlling my behaviour and the behaviour of my students can be done, what behaviours I use, what behaviours I don't use and why. The second open-ended statement was, "I believe the most important things the foreign language teacher can gain from knowing the Flanders system are"; and the responses were as follows: An understanding of how to elicit student responses, original ideas and their feelings, how to get the students to contribute in class without fear, how to develop and use behaviours which accept encourage and praise the student, the importance of having some tangible checklist to analyse behaviour in class, the need for more and varied encouragement so as not to sound trite with merely "bien", the basics of teaching no matter what the subject, how to react to student responses, varying teaching techniques. 248 Revista de Lenguas para Fines Específicos N™ 5 y 6 (1998 y 1999) Towards greater objedivity in classroom observation Hill, B. 1982. Making the Most of Video. London, CILT. Jarvis, G.A. 1968. A Behavioural Observation System for Classroom Foreign Language Skill Acquisition Activities. Modern Language Journal 52: 335341. Klapper, J. 1991. The Role of the Video Camera in Communicative Language Teaching and Evaluation. Language Learning Journal, No.4, Sept., 1991. Krashen, S. 1981. Second Language Acquisition and Learning. Oxford, Pergamon Press. — 1983. The Natural Approach. Oxford, Pergamon Press. Lonergan, J. 1990. Making the Most of the Video Camera. London, CILT. Malamah-Thomas, A. 1987. Classroom Interaction. Oxford, OUP. Marsh, C. 1989. Some Observations on the Use of Video in the Teaching of Modern Languages. British Journal of Language Teaching, 27, 1, Spring, 1989. McKenzie, P. 1989. Teacher Appraisal. Victoria, Journal of Australian Education, Review No. 28. McMahon, A. 1992. Teacher Appraisal. Journal of Teacher Development, 1, 1. Medley, D. & Mitzel, H. 1963. Measuring Classroom Behaviour by Systematic Observation In Gage, Handbook of Research on Teaching. 247-238. Chicago, Rand McNally. Moskowitz, G. 1967. The Flint System. In Boyer, Anthology of Classroom Systems (Philadelphia Section) 15, 1-15. — 1968, The Effect of Training Language Teachers in Interaction Analysis. Foreign Languages Annual, 1:218-235. — 7976 «The Classroom Interaction of Outstanding Language Teachers.» Foreign Language Annals 9/2. Nearhoof, O. 1969. Teacher Pupil Interaction in the Modern Language Classroom. Unpublished paper cited by Grittner (op. cit.). Nunan, 1992. Research Methods in Language Learning. Cambridge, Cambridge University Press. Pearson, J. 1990. Putting Pupils in the Picture Using the Video Camera. Language Learning Journal, 2, 71-77. Sinclair, J. & Coulthard, R.M. 1975. Towards an Analysis of Discourse. Oxford, OUP. Ullman, R. & Geva, E. 1984. Approaches to Observation in Second Language Classes. In Brumfit, C. J. (ed.) Language Issues and Education Policies (ELT Documents 119) Oxford. Pergamon. Wragg, E.C. 1987. Teacher Appraisal. Basingstoke, Macmillan Education. Revista de Lenguas para Firmes Específicos N"' 5 y 6 (1998 y 1999) 255