Understanding sampling by Chilean secondary school students

Statistics Education Research Journal, 20(2), Article 11. https://doi.org/10.52041/serj.v20i2.383 © International Association for Statistical Education (IASE/ISI), December 2021 UNDERSTANDING SAMPLING BY CHILEAN SECONDARY SCHOOL STUDENTS KAREN RUIZ REYES Universidad de Granada [email protected] JOSÉ MIGUEL CONTRERAS GARCÍA Universidad de Granada [email protected] ABSTRACT In statistical inference, importance of sampling is recognized as one of its key concepts, which has allowed its incorporation internationally in different curricular guidelines and specifically in the Chilean curriculum, since the first notions of sampling are introduced in 7th Grade. This paper presents an analysis of the responses to an open-ended written questionnaire, designed to evaluate understanding of sampling, that was applied to a sample of 1,241 Chilean secondary students of 8th, 10th and 12th Grades in six different secondary schools. A mixed methodology was used, with qualitative description of responses and a quantitative analysis of their frequencies. The results reflect outstanding difficulties in the use of elements related to sampling and its properties in different problem situations. For example, students can distinguish the concept of sample in contexts close to their experiences; but when faced with different sampling methods, they are not able to identify biases associated to sample selection. Thus, when deciding if a sample is representative, they mostly identify cases in which the given sample is not. Keywords: Statistics education research; Sampling; Understanding; Statistical inference; Secondary education 1. INTRODUCTION Through the analysis of different curricular guidelines from Spain, the United States and Chile (MEC, 2015; CCSSI, 2010; MINEDUC, 2009; 2012; 2015a) it is possible to observe that the teaching of statistics focuses, mainly, on descriptive concepts during the first school years, while inferential topics are being taught to last years of secondary school or to university courses. Nowadays, some mathematics education researchers (Batanero, 2013; Makar & Ben-Zvi, 2011; Ben-Zvi, Bakker, & Makar, 2015) highlight the importance of the foundations of statistical reasoning, which suggests a wider and deeper role of statistics within mathematical school instruction. In relation to samples, Burrill and Biehler (2011) stated that sampling theory studies how samples and their properties are selected to draw unquestionable conclusions. Reaching an understanding of sampling as a fundamental concept in inference is indispensable to study hypothesis testing and confidence intervals. The Chilean Curricular Framework (MINEDUC, 2009, 2012, 2015) establishes that primary education includes eight grades, while secondary education includes four grades. During the first primary school years, topics of descriptive statistics are studied. This includes the use of certain terms as synonyms of sampling, such as data collection or data group to describe the datasets under study, but without linking these to a population. The sample concept is formally addressed in 7th Grade (12– 13 years old) as a subset within the population. Furthermore, concepts such as intuitive ideas of representativeness of sample, its randomness, and the possibility to estimate results from it, are studied. In 10th Grade (15–16 years old) additional concepts are studied, such as sample size, different counting techniques (permutations, variations and combinations) in randomization, with and without replacement. Finally, in 12th Grade (17–18 years old), students make conjectures for the distribution type to which of the samples means 𝑥𝑛 (sized n) tends when n increases, which is an introduction to sampling distribution notion and its relationship to normal distribution (MINEDUC, 2015). Ruiz-Reyes 2 et al. (2017) provide a wider and more detailed description of the concepts related to sampling that are present in Chilean curricular framework, alongside comparison among other international curricular guidelines. Despite the importance given to sampling as one of the elementary concepts of inferential statistics, Ben-Zvi et al. (2015) determined that less attention has been paid to research on the topic compared to other statistical concepts. Therefore, taking into account this information and the early presence of sample concepts in the Chilean curriculum, there is the need to carry out an exploratory study to assess secondary school students’ understanding of sampling, to provide information about the Chilean context, where previous studies on this topic have not been carried out. This paper focuses on the research question: What do Chilean secondary school students understand about sample and sampling? To answer this question, a questionnaire of six open-ended items was elaborated and applied (see Appendix). The results provided by a sample of 1,241 Chilean secondary school students are detailed in the Result section of this report. 2. BACKGROUND Watson and Moritz (2000) evidence that students from 3rd Grade possess very elementary and particular notions about sample, specifically, of ideas derived from daily life experiences with sampling products (e.g., in a supermarket). Also, the students’ conclusions on population were drawn from really small sample data, with no concern for bias in the sample selection method. Students from 6th Grade tended to formulate varying beliefs about the sample size and the sampling method similar to the students from 9th Grade who considered variation in the population; that is, they needed to have a sample that was big and representative enough. However, very often the students did not identify bias associated to the sample selection method. A study on secondary school students’ reasoning about sampling was developed by Watson (2004) through longitudinal interviews over 3 or 4 years with 38 students from 6th to 12th Grade (11–18 years old). This study evidenced that students prefered samples selected out of a biased process; that is, voluntarily delivered purposeful samples, over random samples. The students’ responses, in general, improved gradually from the first to the last interview. From another study, Watson (2004) and Harradine et al. (2011) argued that students had difficulties with the idea of variability in populations, were overly confident with small samples, and did not consider the importance of sample size when drawing random samples. The study performed by Meletiou-Mavrotheris and Paparistodemou (2015) analyzed the informal inferential reasoning of 69 students from 6th Grade (11 years old) about the following concepts: sample, sample size, sampling method, and bias. Eighty five percent of students had heard the word sample in contexts unrelated to school (a shop, a pharmacy, etc.); however, almost half of them (46%) did not explain the concept or gave an incomplete answer. Forty percent of the students provided definitions reflecting the idea of “a part”, without the associated whole that characterizes the sample-population relationship or referred to it as “a test of something”; only twelve percent of the students provided more elaborate answers and recognized the part-whole relationship between sample and population. 3. THEORETICAL FRAMEWORK The guidelines proposed by Zieffler et al. (2008) define Informal Inferential Reasoning (IIR) as: “The way in which students use their informal statistical knowledge to make arguments to support inferences about unknown populations based on observed samples” (p. 44). Additionally, these researchers pointed at three components of IIR to be considered: 1) Making judgments, claims, or predictions about populations based on samples, but not using formal statistical procedures and methods (e.g., p-value, t tests) (p. 45); For example, a researcher might ask whether the student made reasonable inferences about one or more populations based on one or more samples (p. 53). 2) Drawing on, utilizing, and integrating prior knowledge ..., to the extent that this knowledge is available (p. 45); 3 For example, a researcher might ask how the student used and integrated informal knowledge (e.g., everyday knowledge of the problem context, prior knowledge about statistical concepts, real world knowledge and experience, and statistical language) in making inferences (p. 53). 3) Articulating evidence-based arguments for judgments, claims, or predictions about populations based on samples (p. 45). For example, a researcher might ask how the student has used evidence to support his/her arguments in making inferences, and also, how well the evidence used supported the inferences made (p. 53). 4. METHODOLOGY In this study, a mixed method was used (Hernández et al., 2014) to analyze data collected from a questionnaire. The analysis included a qualitative component, since there is a description of the responses provided by the students; and a descriptive quantitative analysis, specifically through tables that summarize the results of main variables considered in this research. Six open-question items—validated by experts—were applied, which were based on some of the studies described previously in the Background section. In Table 1 we assigned the reference to the source from which each question was adapted, including the questions that were prepared by the authors of the present research. Five groups of questions were used (see Table 1), which had the purpose of measuring some of the aspects related to sampling. In total, the questionnaire had 20 questions distributed among the six items proposed (see Appendix). From the expert revision performed, for each question we established the analysis categories a priori, based on the expected correct answer for all the six items of the questionnaire. These will be described next, and their main features will be detailed in the frequency tables in the Results section. Table 1. Reference studies Group Objective Item Source 1 Understand the definition of sample 1 2 Watson (2004) Meletiou-Mavrotheris & Paparistodemou (2015) 2 Suggest a sampling method 3 Prepared by the authors 3 Decide whether a sample is representative 4 Del Valle et al. (2013) 4 Identify the sample and the population 5 Muñoz et al. (2013) 5 Obtain all the possible samples in a finite population 6 Prepared by the authors For Item 1 (see Appendix), it is expected that students are able to explain in their own words what is understood by sample and mention any context in which the concept has been heard. The answer is considered correct if the student states any of the following situations: 1) Yes. It is a part of a bigger whole; 2) Yes. Gives examples in context but does not explain meaning; 3) Yes. When something is given to test. For Item 2 (see Appendix), students are requested to apply the concept of sampling and its representativeness in a contextualized situation, with the purpose to evaluate whether they understand the concept of sample. Correct answers for question 1 include characterizing a sample “as a subset of a population”, or that it corresponds to “a group of Chilean secondary school students.” For question 2, some reasons considered correct can include “because it is difficult to survey everybody”; “because it is an expensive process”; and “because it is time-consuming.” Finally, for question 3, the correct answer for the first part is “No, I don’t agree with selecting only 10 students”; and for the second part they were expected to indicate the sample size they would choose. Item 3 (see Appendix) asked students to suggest a sampling method from a known sample size but based on an unknown population size. The correct answer was mentioning some sampling method and justifying that choice. 4 Item 4 (see Appendix) requested students determine whether the sample indicated was representative or not; in the second part, students justified their choice. In Table 2 correct answers for Item 4 are presented. Table 2. Correct answers Item 4 Sample Is it representative? Why? 1 Yes Because they were chosen randomly 2 No Because only one group of users was surveyed 3 No Because it does not consider variability In Item 5 (see appendix), students must be able to identify the population and sample for each of the provided sets of data. The correct answers for that item are presented in Table 3. Table 3. Correct answers Item 5 Population Sample 1. All yogurt units manufactured 1. A determined number of each flavor 2. The prices of the meat varieties at the butchery 2. The prices of a variety of meats (beef, fish, poultry) 3. All the ants in the insectarium 3. A certain amount of ants from the insectarium 4. All the cities of the country 4. One city per region Item 6 (see Appendix) asks students to write down, all the different samples (with and without replacement) can be get, from a finite four elements population. In Table 4, the correct answers for Item 6 are specified. Table 4. Correct answers Item 6 Samples with replacement Samples without replacement 1-2; 2-1; 3-1; 4-1. 1-3; 2-3; 3-2; 4-2. 1-4; 2-4; 3-4; 4-3. 1-2; 2-3. 1-3; 2-4. 1-4; 3-4. For all the items, any answers that do not fit in the above mentioned criteria will be considered as incorrect. 4.1. PARTICIPANTS AND CONTEXT In this study, the sample was formed by 1,241 Chilean students; 274 (22.1%) from 8th Grade (13– 14 years old); 529 (42.6%) from 10th Grade (15–16 years old); and 438 (35.3%) from 12th Grade (17– 18 years old). The students attended six different secondary schools in the city of Osorno, Chile during the first semester of 2018. Out of these, 132 (10.6%) attended private schools, 411 (33.1%) attended semi-private schools, and 698 (56.2%) attended public schools. The group was comprised of 690 (55.6%) females and 551 (44.4%) males. The students answered the questionnaires in writing. They were given 45 minutes and it was administered as an activity that was part of their regular Maths lessons, with one of the researchers present, who explained the objective of the questionnaire and clarified possible doubts about its completion. It must also be said that, at the time of administering the questionnaire, neither teachers in charge in each class nor participating students were asked if they had received previous instruction on sampling, since this aspect was not part of the research objective. As mentioned above, although the Chilean curriculum includes the sample concept in 7th Grade, we cannot state that these participating students had received prior instruction on this subject. 5 5. RESULTS The results of the study are described through the analysis of the students’ responses to each item of the questionnaire (see Appendix). We present their ideas and the percentages of correct and incorrect answer per question. The diversity of responses given for Item 1 (see Appendix) is detailed in Table 5. A low percentage did not answer (1.5%) or had not heard about that concept (2.1%); while responses from 96% of the students confirmed they had heard the word sample previously. When the students answered that a sample is part of a bigger whole, it was observed that the percentage of the answer increased gradually as the age of the students increased. Although, most students affirmed to have heard the word sample, only 43.8% provided examples in several contexts, mostly without explaining its meaning. Table 5. Analysis of answers Item 1 Class 8th Grade 10th Grade 12th Grade Answers n (%) n (%) n (%) Total No answer. 5 (0.4) 8 (0.6) 5 (0.4) 18 (1.5) No. Never. 7 (0.6) 15 (1.2) 4 (0.3) 26 (2.1) Yes. It is part of a bigger whole. 63 (5.1) 134 (10.8) 166 (13.4) 363 (29.3) Yes. Gives examples in context but does not explain meaning. 133 (10.7) 270 (21.8) 141 (11.4) 544 (43.8) Yes. Gives explanation without any solid basis. 66 (5.3) 81 (6.5) 106 (8.5) 253 (20.4) Yes. When something is given to test. 0 (0) 21 (1.7) 16 (1.3) 37 (3) Total 274 529 438 1241 Comparing answers per class (see Table 6), it was observed that the highest percentage of correct answers corresponded to students from 10th Grade (34.2%), followed by students from 12th Grade (26%); and lastly by students from 8th Grade (15.8%). Table 6. Correct and incorrect answers Item 1 Class 8th Grade 10th Grade 12th Grade Answers n (%) n (%) n (%) Total (%) Incorrect 78 (6.3) 104 (8.4) 115 (9.3) 297 (23.9) Correct 196 (15.8) 425 (34.2) 323 (26) 944 (76.1) Total 274 529 438 1241 In sum, 944 students (76.1%) gave a correct answer by indicating they had heard the word sample in some context (health center, supermarket, shop, pharmacy, etc.) but not necessarily in a Math lesson, which means their responses were based on informal statistical aspects. Conversely, 297 (23.9%) provided an incorrect answer, by not answering or giving examples in unclear contexts. For the first question in Item 2 (see Appendix), students manifested they had a relatively clear idea of the concept of sample in a context that was close to their own reality, given that 27 (2.2%) students did not answer, as seen in Table 7. It is observed that 12.8% answer incorrectly, while 83.8% (1,040) mentioned ideas such as a small part, a percentage, or an amount, and only 15 students (1.2%) give a more accurate definition, that is, the answer was a subset of a population. To sum up, 85% of the students answer this item correctly. The analysis of responses per class shows that 16.9% of students from 8th Grade answered correctly; this percentage increased when considering students from 10th Grade (35.8%) and, similarly, 12th Grade (32.2%). 6 Table 7. Analysis of answers Item 2, question 1 Class 8th Grade 10th Grade 12th Grade Answers n (%) n (%) n (%) Total (%) No answer 6 (0.5) 15 (1.2) 6 (0.5) 27 (2.2) Incorrect 57 (4.6) 70 (5.6) 32 (2.6) 159 (12.8) A part, a small part 153 (12.3) 289 (23.3) 262 (21.1) 704 (56.7) A percentage 16 (1.3) 42 (3.4) 30 (2.4) 88 (7.1) An amount 39 (3.1) 107 (8.6) 102 (8.2) 248 (20) A subset from a population 3 (0.2) 6 (0.5) 6 (0.5) 15 (1.2) Total 274 529 438 1241 In the case of question 2 (see Table 8), 65.5% (812) of students have a clearer idea of the factors that are relevant when choosing a sample from a population; for example, they mentioned ideas about time-consuming processes, costs of the survey, or access to interview the total population. This means that a high percentage of the students took into consideration the factors that affected the selection of representative samples; while 34% (429) answered incorrectly or gave no answer. The main reason mentioned to select a sample is because it is difficult to survey everybody (31.3%); as a second choice, they said that it would be time-consuming (21.2%). Students from 10th Grade were more inclined to choose these answers, followed by students from 12th Grade and lastly, by 8th Grade. Table 8. Analysis answers Item 2, question 2 Class 8th Grade 10th Grade 12th Grade Answers n (%) n (%) n (%) Total (%) No answer 14 (1.1) 22 (1.8) 14 (1.1) 50 (4) Incorrect 104 (8.4) 169 (13.6) 106 (8.5) 379 (30.5) Because it is difficult to ask everyone 79 (6.4) 165 (13.3) 145 (11.6) 389 (31.3) Because it is expensive 2 (0.2) 4 (0.3) 1 (0.08) 7 (0.6) Because it is time-consuming 60 (4.8) 118 (9.5) 85 (6.8) 263 (21.2) It is faster or easier 3 (0.2) 18 (1.5) 41 (3.3) 62 (5) Because the same results can be obtained 12 (1) 33 (2.7) 46 (3.7) 91 (7.3) Total 274 529 438 1241 In the answers to Question 3 of Item 2 (see Table 9), it is observed that 19.8% do not answer; 35.6% disagree with choosing “only 10 students” as a sample; and 44.6% agree with the statement proposed. Table 9. Analysis of answers Item 3, question 3 a Class 8th Grade 10th Grade 12th Grade Answers n (%) n (%) n (%) Total (%) No answer 69 (5.6) 91 (7.3) 86 (6.9) 246 (19.8) I disagree 79 (6.4) 203 (16.3) 160 (12.9) 442 (35.6) I agree 126 (10.2) 235 (19) 192 (15.4) 553 (44.6) Total 274 529 438 1241 For this question, only 35.6% answered correctly. Although, in the previous questions, the students evidenced an elementary level of comprehension of sample and the limitations to perform a study that surveys at total population—when required to quantify the subjects of their sample, their conclusions were not really accurate. It could be inferred that these answers were influenced by the fact that the population under study was unknown, which made it hard to quantify and may lead to mistakes when interpreting the question. Again, it was observed that students from 10th Grade are the ones who 7 answered this question best, followed by 12th Grade; and lastly, 8th Grade. For the second part of question 3, How many would you choose? (see Table 10), 34.3% of the studnets indicated they would choose “10 students”, just like the first part of the question stated. Another 36.8% of the studnets pointed out that they would consider “more students” depending on the total number of interviewed students, or the total number of schools involved to the study; and only 9.2% consider that all Chilean secondary school students should be interviewed, which showed they did not use the reasoning provided in the previous question about selecting a representative sample. It was observed that 56.6% of students affirmed that a sample bigger than 10 students should be selected (considering all the reasons presented); although they do not refer expressly to representativeness. Some students some considered the sample should be selected randomly, or according to the total number of interviewed students. We observed that 22.8% of students from 10th Grade; 21.3% from 12th Grade, and 12.5% from 8th Grade answered the question correctly. Table 10. Analysis of answers Item 3, question 3 b Class 8th Grade 10th Grade 12th Grade Answers n (%) n (%) n (%) Total (%) No answer 15 (1.2) 29 (2.3) 23 (1.9) 67 (5.4) 10 students 91 (7.3) 198 (16) 137 (11) 426 (34.3) More students 96 (7.7) 187 (15.1) 174 (14) 457 (36.8) 50% 13 (1.1) 27 (2.2) 21 (1.7) 61 (4.9) All of them 35 (2.8) 45 (3.6) 34 (2.7) 114 (9.2) Depends on total 11 (0.9) 22 (1.8) 35 (2.8) 68 (5.5) Randomly 0 (0) 1 (0.1) 2 (0.1) 3 (0.2) Less than 10 13 (1.1) 20 (1.6) 12 (1) 45 (3.6) Total 274 529 438 1241 When students were asked to suggest how they may select a sample (with a sample size given), from a contextual situation close to their reality (Item 3, see Appendix), it was observed that 38.9% of them mentioned a sampling method, but do not justify its choice; and 5.7% refered to selecting a biased sample; for example, they chose a group belonging to a particular school level, or considered other common characteristics, such as sex or age of the participants (see Table 11). Overall, Item 3 shows a high percentage of incorrect answers (59.3%); only 40.7% of students answer correctly, indicating a sampling method and justifying its choice. When the results are organized forr this item per class, it was noted that students from 10th Grade (22.2%), followed by 8th Grade students (9.8%) and, finally, by 12th Grade students (8.7%) answered correctly. It can be argued that the highest difficulty to answering this question arises from not indicating explicitly the total number of students of the school (population) in the statement, which may have prevented them from understanding the situation stated in the question. Additionally, as the last curricular changes for 10th Grade do no longer include sampling methods (MINEDUC, 2015), this might be a factor that influences the high percentage of incorrect answers. Table 11. Analysis of answers Item 3 Class 8th Grade 10th Grade 12th Grade Answer n (%) n (%) n (%) Total (%) No answer 32 (2.6) 101 (8.1) 49 (4) 182 (14.7) Mentions a sampling method, does not justify answer 112 (9) 124 (10) 247 (19.9) 483 (38.9) Mentions obtaining a sample from a biased group (some particular feature) 8 (0.7) 29 (2.3) 34 (2.7) 71 (5.7) Uses any sampling method and justifies answer 122 (9.8) 275 (22.2) 108 (8.7) 505 (40.7) Total 274 529 438 1241 8 Item 4 (see Appendix) was adapted from an activity extracted from a textbook (Del Valle et al., 2013, p. 247) in which students were to indicate if the given sample was representative or not, as well as to justify their responses. In Table 12, the correct answer percentages are presented to summarize the data. Table 12. Analysis of answers Item 4 (correct answers) Class 8th Grade 10th Grade 12th Grade n (%) n (%) n (%) Total (%) Sample 1. 118 (9.5) 193 (15.6) 165 (13.3) 476 (38.4) Sample 2. 99 (8) 217 (17.4) 198 (16) 514 (41.4) Sample 3. 114 (9.2) 213 (17.2) 200 (16.1) 527 (42.5) In the first sample, only 38.4% (476 students) answered correctly, pointing out that the sample is representative, in a situation that involves a random sampling method. In the second one, only 41.4% (514 students) answered that the given sample is not representative, because it is a restricted sampling method, which implies that subjects are chosen by biased process and it is not a representative sample of the population. Finally, in the third sample, only 42.5% (527 students) answered that the sample is not representative, since it used a self-selected sampling method, which showed that subject selection choice poses problems for identifying whether it is representative or not, as it considers subjects with specific characteristics, without considering all subjects of population under study. It was observed that the students recognized mostly non-representative samples; but in the first sample they do not identify that it is a random sampling method. They focused only on the given sample size and did not consider relevant that it had been selected randomly, resulting in incorrect answers (61.6% or 765 students). Among the explanations to justify the answers to the first sample, students mentioned the following: because a third of the population was chosen; because it was a random sample; and half of the population should be chosen. For the second sample, they point to reasons like: they should survey more people; because only one group of users are surveyed; and they should be surveyed at different times. Finally, for the third sample, the explanations indicated are: because the diversity of heights should be considered; variability was not considered; and all students should be measured to get an average. Item 5 (see Appendix) was adapted from an activity from a textbook (Muñoz et al., 2013, p. 275). The item required students to indicate the population and a possible sample for each statement. In Table 13 the percentages of correct and incorrect answers are presented to facilitate the readers’ analysis. The majority of students answered Item 5 incorrectly (see Table 13), both to identify population and sample requested, showing that they had problems recognizing those concepts in contexts far removed from the school reality. In addition, they do not identify the difference between the part-whole relationship in the contexts mentioned in the item statement. From the analysis of responses to Item 5 per class, it was observed that 12th Grade students obtained better results in identifying population, which decreased in the other classes. The same phenomenon occured for the fourth sample; whereas for the first three samples, those who obtain more correct answers were 10th Grade students, followed by 12th Grade students. The students who obtained the lowest results were those in 8th Grade. Table 13. Analysis of answers Item 5 Population Sample Answer Answer Incorrect n (%) Correct n (%) Incorrect n (%) Correct n (%) Data set 1. 856 (69) 385 (31) 897 (72.3) 344 (27.7) Data set 2. 976 (78.6) 265 (21.4) 974 (78.5) 267 (21.5) Data set 3. 785 (63.3) 456 (36.7) 879 (70.8) 362 (29.2) Data set 4. 1114 (89.8) 127 (10.2) 1149 (92.6) 92 (7.4) 9 Finally, in Item 6 (see Appendix), only 22.4% answered this question correctly, writing both list of samples requested; 25.8% gave partially correct answers, writing only one of the list of samples requested (i.e., writing the sample with replacement or the sample without replacement); 14% wrote the sample with (or without) replacement incompletely, with some missing elements; and 37.8% did not answer (see Table 14). When analysing the results from Item 6 by correct answer and class, it was perceived that a higher number of 10th Grade students answer correctly (11.4%), followed by 12th Grade students (7.7%); and lastly by 8th Grade students (3.3%). Since this question involved combinations and permutation, we observed that some 12th Grade students answered incorrectly because they confused the formula of the respective counting technique that should be used, as they do not identify that this formula only gives the number of elements that each sample should contain. Table 14. Analysis of answers Item 6 Class 8th Grade 10th Grade 12th Grade Answer n (%) n (%) n (%) Total (%) No answer 104 (8.4) 221 (17.8) 144 (11.6) 469 (37.8) Writes list of samples with replacement 70 (5.6) 86 (6.9) 116 (9.4) 272 (21.9) Writes incomplete list of samples with replacement 24 (1.9) 20 (1.6) 29 (2.3) 73 (5.9) Writes list of samples without replacement 10 (0.8) 22 (1.8) 16 (1.3) 48 (3.9) Writes incomplete list of samples without replacement 25 (2) 38 (3.1) 37 (3) 100 (8.1) Writes both lists of samples 41(3.3) 142 (11.4) 96 (7.7) 279 (22.4) Total 274 529 438 1241 (100) 6. DISCUSSION The findings of this exploratory study have allowed us to identify and characterize what Chilean secondary school students understand about samples and sampling, from the perspective of Informal Inferential Reasoning (IIR). Students, at the moment of elaborating their arguments, do not necessarily use formal statistical methods, in addition, their answers may (or not) contain concepts or formal statistical language (Zieffler et al., 2008), as it has been exposed in the analyses presented in the Results section. The responses obtained by the sample of 1,241 Chilean secondary school students provide useful information on their understanding of sampling, as well as on the main difficulties related to it. The results indicate that students distinguish the sample concept in contexts close to their experiences (health, science, shopping, etc.), with very few students giving more elaborate responses when connected to a mathematical context; that is, they use the randomness concept in their arguments. Only fifteen students defined sample as a subset of the population. Similar results were obtained in the studies by Watson and Moritz (2000) and Meletiou-Mavrotheris and Paparistodemou (2015). Another aspect to consider is that students are susceptible to insensitivity to sample size (Tversky & Kahneman, 1974), that is, they assume that the selected sample, regardless of its size, always represents the population to which it belongs, which induces serious errors of interpretation in statistical analysis. Furthermore, as Kahneman et al. (1982) indicated, this type of person also believes in the law of small numbers; in other words, they believe that when selecting samples that are not large enough, the sample distribution is distributed in the same way as the distribution of the population, regardless of the sample size. When we asked the students to identify the sample of a study in a context closer to their reality, some inconsistencies in their responses were observed. For example, they were able to identify the limitations that may lead a researcher to survey all subjects in a given population, but when they were to suggest a sample or when we asked what sample size they would choose, 43.4% of the students in our study responded incorrectly. When the students tried to suggest a sampling method, only 38.9% gave a response, but did not justify their selection method. It should be noted that most students do not provide a sampling method