Justifying the use of Internet sources in school assignments on controversial issues

PUBLISHED QUARTERLY BY THE UNIVERSITY OF BORÅS, SWEDEN VOL. 23 NO. 1, MARCH, 2018 Justifying the use of Internet sources in school assignments on controversial issues Teemu Mikkonen Introduction. This study concerns students’ criteria in the evaluation of Internet sources for a school assignment requiring reflections on a controversial issue. The findings are elaborated by analysing students’ discursive accounts in justifying the use or non-use of sources. Method. The interview data was collected in a Finnish upper secondary school during classes on religion and ethics. Thirty-nine students were interviewed in one to three person groups after they had given the presentation in the classroom. Analysis. The interviews were analysed using the discourse analysis in social psychology approach. The analysis concerned how different accounts related to information evaluation are used to justify the truthfulness of knowledge. Results. The most used evaluation criteria were the authority and neutrality of information. Various types of evaluation criteria and arguments were used simultaneously. The arguments were, in part, comparable to evaluation criteria presented in previous research. Conclusion. The evaluation criteria are cultural objects that can be utilized in various ways within different discursive contexts. The criteria were used in students’ accounts alternately, not as mutually exclusive. Personal interests and high motivation with regard to the school assignment’s subject were essential for students to identify and use more reflective and diverse argumentation related to evaluation criteria. Introduction The information environment in schools is changing rapidly. The growing number of networked sources calls for using better and more explicit methods to evaluate information that is available on the Internet. For example, previous research has recommended that students use various checklists, which helps them evaluate information (e.g., Kapoun, 1998 ; Alexander and Tate, 1999). These checklists have been criticized in several studies, but they still form the basis for education on evaluating Internet sources in schools (e.g.,Meola, 2004; Metzger, 2007). In this paper, the role of the items included in checklists and other information evaluation criteria are analysed as part of the students’ accounts about information evaluation tasks. This perspective provides the possibility to discover the processes behind criteria that students actually refer to while seeking information for their school assignments. In Internet-based information environments, argumentation related to the identification, selection and use of different information sources becomes more important, especially in the context of selfgoverned school assignments. This paper empirically examines how students evaluate and justify the use of Internet sources in a class on religion and ethics. In this class, students were assigned to make a presentation from controversial topics of their choice. Because topics were controversial in nature, the justification of choices was critically important. The study of argumentation related to information evaluation promotes the understanding of how students identify and use arguments related to the evaluation of Internet sources. Consequently, the results have practical implications for teaching critical evaluations of information in schools. Further, it also presents an example of how justifications of evaluation criteria employed by the students can be investigated using the discourse analysis in social psychology research method. The overall aim of the present investigation is to specify how students evaluate and justify the use of Internet-based information in school assignments dealing with controversial issues. To this end, the study addresses the following research questions: RQ1. What kind of criteria do students employ to evaluate information available in the networked sources? RQ2. What kind of discursive accounts do students provide to justify the use or non-use of such sources? The rest of the paper is organized as follows. First, a short background review of research related to evaluation criteria and judgement factors is presented. Second, discourse analysis in social psychology is characterized as a research method. Third, the findings are reported by specifying the criteria and the arguments by which the students evaluated the Internet sources. Fourth, drawing on discourse analysis, the ways in which such criteria were constructed as meaningful are specified by focusing on diverse accounts presented by the students. Finally, the findings are discussed and some conclusions are presented. Background There are a number of studies examining the use of Internet sources with a focus on evaluation criteria and judgement factors (e.g., Metzger, 2007; Rieh, 2002; Meola, 2004). Kapoun (1998) has presented a checklist of five criteria for evaluating Internet sites: accuracy, authority, objectivity, currency and coverage. Variations of this checklist model have been the basis for librarians to teach the evaluation of Internet sites (Alexander and Tate, 1999). Metzger (2007) summarized previous research related to the checklist model. Her conclusion is that not all of the five checklist criteria are used in everyday information seeking. Further, most respondents used a criterion—Website appearance—that is not included in the list at all. Meola (2004) criticized the checklist model, because it is unwieldy for users to perform and is unrealistic regarding teaching critical evaluation. He offered a model of contextual credibility assessment that emphasized the external characteristics of Websites. Rieh (2002) identified and characterized the facets of information quality based on users’ own words, and received results that were different from previous studies. Her study showed that quality and authority are important issues for users when they are interacting with information in an uncontrolled environment such as the Internet. The authority of the sources was closely related to the quality of the information. Metzger (2007) introduced a dual processing model of credibility assessment, which was based on Chen and Chaiken’s (1999) dual processing theory. The model emphasizes the importance of motivation in information evaluation by proposing that the evaluation of Internet sources depends on users’ motivation for seeking information. Here, the focus is on argumentation that occurs after information sources have been selected and used in students’ schoolwork. This perspective is different from the studies reviewed above. In the checklist model, contextual credibility assessment, and analysis of cognitive authority, the focus is on the analysis of decision-making situations before the use of Internet sources. The argumentation on the qualities or choice of information sources afterwards is not studied (see Metzger, 2007; Meola, 2004; Rieh, 2002, p. 146). When students are required—by the school assignment or by an external researcher—to argue for the information sources they have used, they need to reconsider the quality, authority and topicality of the sources, especially since there is no general quality control mechanism on the Internet. The retrospective evaluation of information sources is an integral part of students’ accounts that they employ while explaining why they have used certain information sources. The analysis of the accounts related to information use on the Internet will give new insights and complementary information to the research of evaluation criteria and judgement factors. Method Discourse Analysis The goal in the discourse analytic perspective is to explore the possibilities of language use in different contexts (e.g., Talja, 1999). In the late 1990s, discourse analysis was introduced to library and information science as a method that allows a wider inclusion of context dependency and variability in qualitative interviews (Talja, 1999). The discourse analytic perspective concentrates on the regularities of language use: the kinds of descriptions and accounts of a topic that are possible, the type of evaluations they are based on, how the different modes of accounting construct different versions of the topic or produce different kinds of truths, and the accomplishments of these versions (Talja, 1999; Wetherell and Potter, 1988 ). Discourse analysis contains various approaches. The present study draws on the ideas of discourse analysis in social psychology, which is based on the writings by Jonathan Potter and Margaret Wetherell (Potter and Wetherell, 1987; Wetherell and Potter, 1988; Potter, 1996; see also McKenzie, 2003). Discourse analysis in social psychology is a mixture of methods and theories (including ethnomethodology, semiology and conversation analysis) and it ‘seeks to incorporate insights from a variety of discourse analytic approaches’, (McKenzie, 2003, p. 268). In the present study, discourse analysis in social psychology provides a methodological approach, which helps to interpret the different justification processes as part of social practices (see Potter, 1996). The analysis of the argumentation related to information evaluation helps to understand the process behind the everyday use of different criteria. Discourse analysis in social psychology proceeds from the assumption that there is no speech, text or any other communication free of argumentation. Arguments are acting somehow to accomplish something. From this perspective, for example, concepts like identity and fact are interpreted differently than in everyday contexts. Identities are not understood as pre-existing mental states. Rather, they are subject positions that are constructed during the conversation. Similar to identities, facts are understood as constructed descriptions. Facts are used in conversation to convince other debaters that the speaker is reporting true knowledge presented as an objective description of reality (Potter, 1996; Billig, 1996). Validity and reliability in discourse analysis are interpreted differently than, for example, in quantitative research. In discourse analysis, the criteria of validity and reliability are related to the specimen perspective. In the specimen perspective, the research data is not taken to describe reality. Rather, research data is the specimen of interpretative practices. The reliability of the findings depends on the verifiability of the researcher’s interpretations and they must, in a consistent and identifiable way, be based on the research data (Talja, 1999; Potter and Wetherell, 1987). In the present study, the use of text extracts as the specimens of the data strengthens the reliability and validity of the interpretations in that the extracts provide concrete examples of how the students evaluated information available in the networked sources. In addition, through the texts extracts, the reader can evaluate how consistently the researcher has interpreted the research data. Data collection The empirical data were gathered in autumn 2012 from an upper secondary school situated in the city of Tampere, Finland. The study was focused on a class on religion and ethics—an obligatory course for third grade students. In this class, all students were divided into groups and each group chose a topic on which they made a presentation. Argumentation was an important part of this class, because most topics were controversial in nature. The topics included, for example, medical ethics, women’s rights, gay rights, divorce, ethical consumerism, euthanasia and genetic engineering. Students made online research about topical issues, and discussed them first in their own group and then with the whole class under the guidance of the teacher. In this context, it was appropriate to examine the role of argumentation in information seeking and especially the justifications of the use or non-use of Internet sources on controversial issues. Students prepared their presentations in twenty-one groups of one to three members. There were twenty-one student groups that were interviewed after their presentation. The total number of students was forty-two: twenty-five girls and seventeen boys. The interviews were conducted in groups after the assignment was completed. All except three groups participated as full groups to the interviews. One girl and two boys could not participate. The interviews were recorded and they lasted approximately fourteen minutes a group. All interviews were transcribed before the analysis. Two researchers conducted the interviews using a structured interview guide (see Appendix). The interviews consisted of five themes: motives, group work strategy, information-seeking practices, conception of information sources and the justification of contradictory views. In addition to interviews, the research data consist of the source notes of the group meetings, which students had two to three times before the presentation. In the source notes, students described which source of information they did and did not use. They also had to describe why they decided to use particular sources. The teacher did not assign the source notes and it was voluntary for the student groups to fill in and return the structured forms. The source notes were done in groups, and ten student groups (altogether twenty students) returned the source notes. The interviewers used the source notes to present as much concrete questions as possible. The source notes were also used to support the analysis of the interviews when it was difficult to trace from the transcriptions which or why particular sources had been used. In the present study, discourse analysis in social psychology is used to analyse the ways in which the students provide accounts of criteria whereby they evaluate the credibility and relevance of Internet sources for the needs of school assignments that focus on controversial issues. The context of the assignment persuades the students to justify using particular Internet sources in many ways, and therefore it is appropriate to study the argumentation processes. The identification of different arguments is bound to the context of where they were expressed. This means that the endpoint of the analysis is not, for example, a list of repertoires or rhetorical strategies, but a demonstration of how the different arguments are linked to the social practices, and to the perspective in which they were created (see Savolainen, 2004). Identification of discursive accounts The role of text extracts is different in discourse analysis than in other qualitative research methods. Text extracts are not seen as descriptions of the object of research; they are the objects of research (Talja, 1999) and provide the linguistic evidence for the researcher’s interpretations. Because the aim is to explore the possibilities of language use, the focus is not in generalizability, but rather in how the phenomenon can be seen or interpreted. (Talja, 1999; Potter and Wetherell, 1987). Discourse analysis in social psychology emphasizes two features of the discourse analysis: the epistemological orientation and the action orientation. The epistemological orientation focuses on analysing the ways that discourses are constructed and made to appear factual, and the action orientation focuses on the discursive functions that accounts are meant to perform (Potter, 1996). In the present study, the main emphasis is laid on the action orientation of discourse analysis, because it focuses on students’ accounts that justify the use or non-use of a source type, that is, Internet sources in school assignments. However, the epistemological orientation of discourse analysis is also relevant. The accounts that justify the use or non-use of Internet sources draw on the evaluation of the information’s credibility available in them. Thus, the ways that students convinced others (action orientation) and constructed information as factual or credible (epistemological orientation) with different criteria were intertwined (Potter, 1996). The discursive accounts provided by the students were identified by scrutinizing the transcribed interviews several times until certain themes could be discerned. The most commonly used themes were named, and the text extracts were chosen to illustrate the findings and provide linguistic evidence of the accounts. The extracts were selected on the basis of their density, thoroughness, accuracy and depth of explanation. Each text extract refers to a group (e.g., Group 2) to which the interviewed students (e.g., Mathilda and Olga) belonged. The students’ names are pseudonyms to protect their identities. The checklist model augmented with the criterion of Website appearance has been taken into account when the discursive accounts were categorized. Also the comparison and corroboration techniques from Meola’s contextual approach have been included in the categorization. Furthermore, the use of the concept of cognitive authority defined by Wilson (1983) was observed when the category of authoritativeness of information was analysed. Finally, the relevance of motivation in information evaluation was taken into account when the discussion and final conclusions were made (see Metzger, 2007; Chen and Chaiken, 1999). While specifying the discursive accounts of evaluation criteria, quantitative measures such as about half of the groups (ten out of twenty-one) were provided to substantiate the background information. Most importantly, these measures indicate the popularity of individual evaluation criteria among the students. Results Overall, the students justified information available in the networked sources by employing diverse arguments that were bound to the context of a school assignment. First, the findings specify the use of different evaluation criteria in students’ discursive accounts. Next, the findings reported in this section indicate how the evaluation of networked sources is part of the representations and justifications in the students’ speech. Discursive accounts of evaluation criteria The criterion that the student groups used most in their accounts was based on the authority of the information source. For example, if the information source was an authoritative writer or institution, it was seen as credible. Almost all groups (twenty out of twenty-one) used this criterion in their accounts. Another common criterion was to use the neutrality of the information source as a justification of its truthfulness. In this account, the student was claiming that the source was nonbiased, or if the sources were biased, different views were presented. This criterion was used in almost all groups (nineteen out of twenty-one). About half of the groups (ten out of twenty-one) used their own experience and prior knowledge as an account to justify the truthfulness of the source. Arguably, these groups were motivated to do their school assignment well, because the topic was somehow connected to their personal interests. The accounts in this category were more reflective and critical than in other categories. Arguments including accounts that justify the truth by saying that there were multiple sites with similar information were categorized under the category consensus. About half of the groups (ten out of twenty-one) used this explanation. The visual appearance of the information source was also used as a justification in about half of the cases (ten out of twenty-one). The student groups, who in an interview said that they had less motivation to get good grades, used this explanation more often than others. This account was used mostly along with other accounts. There were also criteria, like freshness or the viewpoint of the site, which were only used in some cases (four out of twenty-one). Authoritativeness of information source In almost all groups, students argued for using a particular information source by referring to the characteristics related to the authoritativeness of the source. This could mean an authoritative institution, author, publication, organization or some other information source that they considered credible (e.g., Rieh, 2002). The following quotation is an example showing how an official institution acts as a self-evident warrant about the credibility of information. Interviewer: How did you evaluate these? Why exactly were these useful? Was it just that it was easy to find? Paul: No, it was the speech. Because it was the former sports minister and… In any case, I didn’t bother to take anything from blogs and any other single person’s points of views that … to form the overall picture that could be a little one-sided. (Group 21) At first, the interviewer asks why students have used these exact sources, and suggests that the reason could be that those sources were easy to find. Behind this suggestion is an account that interviewees have made before to explain the use of certain sources. Nevertheless, the interviewee does not use this as an explanation. The student, Paul, says that the reason why they used this source was that it was a speech by a former sports minister. The sports minister, a representative of an official institution, is used as evidence to justify the truthfulness of the information. Even though the information is in a format (blog) that has, in other contexts, been considered unreliable, in this case the official title and institution warrants the credibility. Therefore it is not seen as ‘a single person’s point of view’, although the information is contained in the former representative’s personal blog post. The next text extract is another example of how authoritativeness is used to justify the credibility of an information source. It demonstrates how academic role models can act as a self-evident testimony to the truthfulness of knowledge. Mathilda: Well, maybe because I’m so interested in [Jacques] Lacan it came to my mind ... because I couldn’t find anything from the net. I had to find books and read about Lacan ... and also from Julia Kristeva, I got the kind of symbolic register that is to say the points of view related to liberal feminism, which were not new, but … that information, which could have changed my own views, maybe. Interviewer: Did you find any … books have a little different view, but any books, which could have had claims that are opposite to yours? Mathilda: Yes, there was. There were a few. I can’t remember who, but some Finnish male researchers who had at some time in the [19]80s made the research; it was certainly fun. (Group 2) Mathilda takes the floor and talks fast over the others about her own interests. She introduces her knowledge about Jacques Lacan and Julia Kristeva. Here and later during the interview, the talk about Jacques Lacan and Julia Kristeva acts as a supposedly credible argumentation that the student uses to convince others about the truthfulness of knowledge. In this quotation, Jacques Lacan and Julia Kristeva belong to the category of authors that is seen to be trustworthy. The interviewee uses the specific vocabulary that is used in the works of the authors, and the use of the vocabulary can be read as a way to stand out from her peers by convincing the interviewers about the knowledge in these academic theories. Jacques Lacan and Julia Kristeva are authorities that the interviewee uses to stress the distinction between book-based knowledge and Internet-based knowledge. First the student refers to the relation of Jacques Lacan and book-based knowledge when she says ‘because I couldn’t find anything from the Internet. I had to find books and read about [Jacques] Lacan’. In this way she expresses that she is accountable, because she uses books as a main information source, but also expresses that this kind of information is not Internet-friendly. The avoidance of Internet-based information can be read as a distinctive feature that helps a student stand out from the others. When the interviewer asks about the opposite claims, the interviewee answers that there are ‘some Finnish male researchers who had at some time in the [19]80s made the research; it was certainly fun’. In this quotation, the interviewee sees that the claim does not need any specific authors to be credible. The mention of ‘Finnish male researchers from [19]80s’ is enough. In contrast to Jacques Lacan and Julia Kristeva, the negative examples will not need any specific names or use specific vocabulary to convince listeners about the knowledge the interviewee has on the work of the authors. The vague expression in this context works as an example that will not need any further explanation. Neutrality of information source Another evaluation criterion, which was used as part of an argument, was the neutrality of the information source. In almost all groups students used this evaluation criterion when they wanted to convince the interviewer that they had used information sources from different sides equitably without taking only the sources that support their own views. In the following text extract, neutrality acts as a self-evident evaluation criterion and as an argument to justify the truthfulness of information. Still, the interviewee wants to bring up her own point of view. Interviewer: Did you find any views from the Internet or books, which were different from yours, and if you did, from where mostly? Olga: I found about the ordination of women. I support the ordination of women, and then from the Internet it is possible to find those who are against the ordination of women. Interviewer: How did you take them into account to this? Olga: Well, they have to be taken into account, because they are an important part of the presentation and I can’t leave them out because they have an opposite point of view. And I tried to speak about them ... even though I thought that this is so wrong, but after a while I got used to it and I could look at it more objectively. (Group 2) The interviewer persuades the interviewees to give their own views by asking which opposite opinions they’ve found from the information sources. An interviewee, Olga, follows this and tells her view about the ordination of women. She sets herself in the defensive position when she says that even if she wanted, she couldn’t ignore opposite opinions. Thus, she distances herself to be an objective observer who must notice opposite views. The reference to her views with the expression ‘this is so wrong…’ is mitigated by saying it with a smile. In this way the contrast between her own opinion and the neutrality is softened. The neutrality is accompanied by the obligation to acknowledge opposing views, even though the identity of the interviewee is (self-) categorized as biased. Therefore, the neutrality is strengthened by professing her point of view. Olga was pointing to neutrality as an obligatory viewpoint that is more binding in this context than her own point of view. Neutrality as an obligatory feature of discourse is the framework in which the student has to act in the school, even though she wants to bring up her own point of view. Students’ own experience and prior knowledge Some students had studied the subject before and were interested in studying it more closely. They had discussed it with different people, and may have had certain opinions about it. In the following quotation, the student uses her own prior knowledge and values as an evaluation criterion and an account. Interviewer: Could you say about these, which are the most essential sources? Were there some particular sites? Mary: Well then, I can’t… It is impossible to say, because anyway, those all… These values are discussed over a long period that it is difficult to tell any exact site, because it was… Because I’ve had so much information already about this case. That is the reason why we hadn’t really sought that [information]... (Group 5) The interviewer asks about the most essential information sources and the interviewee, Mary, does not answer this question, but begins to talk about values. She ties her values to her prior knowledge, which is the reason why she cannot name any exact source. The knowledge she brings to the school assignment is a result of values that are discussed over a long period of time. Mary is reflecting her own position as a person who is interested about the things that this presentation is about, and that is why she does not have to seek information or remember any exact sites. In the next quotation, she continues to explain which kind of sites she has browsed, and how most of the sites are in some way biased. Mary: Or, like I’ve searched information from somewhere; those sites are biased in some direction depending on which side they are… Like for example, some animal rights’ sites or Greenpeace. So they present their case from only one point of view. That, it presents facts, but not necessarily everything. They won’t give those arguments that are against their own. So you have to be critical with these. (Group 5) Mary explains that she has searched information from biased sites like animal rights’ organizations and Greenpeace, but she also brings up her understanding of the viewpoints that these Internet sites have. She explains that those sites might give you one-sided arguments, and that is why she also calls for criticality. This is an example of how the recognition of a student’s own interests can support the understanding of others’ interests and points of view. Thus, her personal interests act as a tool to interpret and evaluate the partiality of networked sources. Appearance of information source In some student groups, the appearance of the Internet sites was a common evaluation criterion that students used when they justified the non-use of certain information sources. In the next quotation, a group of two boys explains which kind of Internet sites they avoided. Interviewer 1: Well, have you discussed together that credibility? Have you evaluated it? Interviewer 2: Generally about these things… Matt: Yes, we have. I found an Internet site and there was a tank drawn nicely with [Microsoft] Paint, and a little bit of small text, and some Harry from the Hervanta neighbourhood has written. Then we noticed that it wouldn’t be from here. David: It is possible to see it from its general appearance then. (Group 18) When the interviewers ask if the students have discussed the credibility of the sources, the interviewee, Matt, answers that they have, and he illustrates by giving an example of a site that they have not used. The illustration consists of appearance details that the interviewees think are selfevident characteristics of a non-credible Internet source. This example shows how a site’s appearance is used as an argument to exclude certain sites. This explanation was often used, particularly to exclude non-credible Internet material. The groups that said they were not motivated to get good grades and just wanted to get the job done used more explanations related to the appearance, freshness or other factors that are external to information content. The groups, who said in the interview that they were motivated to do a good presentation, used this argument in most cases along with other arguments. Consensus between information sources The similarity of information occurring in multiple sites is named the consensus between information sources criterion. It refers to accounts in which students think that there is a consensus of truthfulness in different information sources. This is the main reason why students in the next quotation trust certain Internet sources. Interviewer: How about these sources? Do you think they are trustworthy? Lucy: Well those, which we sifted through, which were slightly unreliable sources, those were surprisingly correct. Interviewer: Okay. How did you evaluate that those were reliable? Lucy: We were just looking—that it is possible to find the same information from multiple sites. (Group 9)