Attitudes towards mathematics at secondary level: development and structural validation of the scale for assessing attitudes towards mathematics in secondary education (SATMAS)

Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 557 - http://dx.doi.org/10.14204/ejrep.40.15163 Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Lara Yáñez-Marquina1, Lourdes Villardón-Gallego 1 1 Faculty of Psychology and Education, University of Deusto, Bilbao Spain Correspondence: Lara Yáñez-Marquina. Faculty of Psychology and Education, University of Deusto. Avda. de las Universidades, 24. 48007 Bilbao (Spain). E-mail: [email protected] © Education & Psychology I+D+i and Ilustre Colegio Oficial de la Psicología de Andaluacía Oriental (Spain) Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 558 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 Abstract Introduction. In secondary education, students’ low achievement and engagement in mathematics are closely related to their attitudes towards the subject. Despite the international body of research, an exhaustive literature review of the existing instruments for measuring it draws attention to the inconsistency in the definition and corresponding factor structure for the construct attitudes towards mathematics. Therefore, the aim of this paper is to develop and validate an instrument for measuring secondary students’ attitudes towards mathematics based on a preliminary detailed theoretical framework. Method. The sample comprised 792 students, with an average age of 13.96 (SD = 1.09) years, from Biscay (Basque Country Autonomous Region, Spain). Confirmatory factor analyses were conducted to test the theoretical proposed non-hierarchical structure, consisting of three first-order factors (student’s math self-concept, perceived usefulness of mathematics and interest for mathematics). Results. The results largely confirmed that this model showed a good fit to the data. Internal consistency, discriminant validity and criterion-related validity tests yielded good Cronbach’s alpha coefficients, strong correlations between the proposed dimensions and moderately positive correlation scores between attitudes towards mathematics and students’ mathematical performance measured with a math achievement test developed ad-hoc. Discussion. The resulting 19-item scale may represent a psychometrically sound instrument both for research purposes and for educational interventions. To conclude, the contribution of the present study to the research on attitudes towards mathematics is discussed, and some issues are suggested for future research to address. Keywords: Attitudes towards mathematics, secondary education, mathematics education, confirmatory factor analysis. Reception: 11.28.15 Initial acceptance: 12.14.15 Final acceptance: 10.16.16 Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 559 - http://dx.doi.org/10.14204/ejrep.40.15163 Resumen Introducción. El bajo logro y el escaso compromiso en matemáticas de los estudiantes de secundaria están estrechamente relacionados con sus actitudes hacia la asignatura. A pesar de la amplia investigación existente al respecto, una revisión exhaustiva de la literatura sobre instrumentos para su medición concluye con la inconsistencia en la definición de la conceptualización teórica y correspondiente estructura factorial del constructo actitudes hacia las matemáticas. Por tanto, el objetivo de este estudio es desarrollar y validar un instrumento para la medición de las actitudes hacia las matemáticas de estudiantes de secundaria basado en un detallado marco teórico previo. Método. La muestra estuvo compuesta por 792 estudiantes, con una edad media de 13.96 (DE = 1.09) años, procedentes de Bizkaia (Comunidad Autónoma del País Vasco, España). Se llevaron a cabo análisis factoriales confirmatorios para testar la estructura teórica no jerarquizada propuesta, consistente en tres factores relacionados de primer orden (autoconcepto matemático del estudiante, utilidad percibida de las matemáticas e interés hacia las matemáticas). Results. Los resultados confirmaron que este modelo presentaba buenos índices de bondad de ajuste. Los análisis correspondientes a consistencia interna, validez discriminante y validez de criterio arrojaron buenos coeficientes alfa de Cronbach, correlaciones significativas entre las dimensiones propuestas y correlaciones moderadas y positivas entre las actitudes hacia las matemáticas y el rendimiento matemático de los estudiantes, que fue medido con un test matemático desarrollado ad-hoc. Discussion. La escala resultante, de 19 ítems, representa un instrumento psicométricamente robusto tanto para fines de investigación como para intervenciones educativas. Para concluir, se discute la contribución del presente estudio al ámbito de las actitudes hacia las matemáticas, y se sugieren algunos aspectos para futuras líneas de investigación. Palabras Clave: Actitudes hacia las matemáticas, educación secundaria, educación matemática, análisis factorial confirmatorio. Recibido: 28.11.15 Aceptación Inicial: 14.12.15 Aceptación final: 16.10.16 Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 560 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 Introduction Students’ mathematical underperformance has become worrisome in many countries (Lipnevich, MacCann, Krumm, Burrus, & Roberts, 2011). Despite the growing importance of mathematical thinking and mathematics-related skills for an individual’s full development in today’s society, this subject is perceived by most students as abstract, difficult, boring and without relation to tasks of everyday life (Ignacio, Nieto, & Barona, 2006). In such a context, socio-cognitive theories have suggested that students’ beliefs and expectations are a major determinant of their pursuit of and engagement in mathematics courses (Crombie et al., 2005; Grootenboer & Hemmings, 2007; Malmivouri, 2007). However, those attitudes are not innate but formed over time by experiences, declining much over the transition from upper elementary school to junior high school (Watt, 2000). An international body of research has highlighted the close relationship between students’ dropout rates in mathematics and both their present and future mathematical performance (Bouchey & Harter, 2005; Anjum, 2006; Skaalvik & Skaalvik, 2006; Samuelsson & Granstom, 2007; Kadijevich, 2008; Williams & Williams, 2010; Lipnevich, MacCann, Krumm, Burrus, & Roberts, 2011). In this line, particularly interesting is the meta-analysis carried out by Ma and Kishor (1997) with longitudinal modelling. The findings suggested that attitudes towards mathematics exerted causal effects on mathematics achievement. However, the corresponding observed effect sizes were found to be small, which was explained by the authors as a consequence of certain psychometric limitations in the instruments designed to measure attitudes towards mathematics. These limits have been also acknowledged in more recent research by Zan, Brown, Evans and Hannula (2006) and Lim and Chapman (2013), which asserted that the factor structure of attitudes towards mathematics remains ambiguous. As seen in Table 1, a deep look on the psychological literature provides a number of differing conceptualizations of the construct attitudes towards mathematics, which has resulted in many instruments targeting to measure it. These measurements for assessing attitudes towards mathematics have been drawn from peer-reviewed articles. Measurements for assessing attitudes towards science or statistics have not been considered in this literature review because recent evidence suggests that attitudes towards these three subjects show different trajectories over adolescence, meaning that each construct should be investigated separately (Barth et al., 2011). Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 561 - http://dx.doi.org/10.14204/ejrep.40.15163 From the existing instruments, the Fennema-Sherman Mathematics Attitude Scales (FSMAS; Fennema & Sherman, 1976) have been the most widely used across all levels of the mathematics curriculum. Since its development, this set of nine subscales has been translated into several languages for its use with samples from different sociodemographic backgrounds. Nevertheless, O’Neal, Ernest, McLean, and Templeton (1988) have yielded poor validity and reliability scores, concluding that the original subscales might not properly gauge the research construct. In line with this, Melancon, Thompson, and Becnel (1994) were unable to find a suitable model fit for the original structural proposal by Fennema and Sherman (1976) and yielded a more parsimonious structure, consisting of eight factors. Likewise, Mulhern and Race (1998) proposed a shortened version of six separate factors, which yielded better internal consistency on both the full scale and underlying subscales. Another interesting instrument is the Attitudes Toward Mathematics Inventory (ATMI; Tapia & Marsh, 2004), which has a more distinct and cohesive factor structure, assessed by both exploratory and confirmatory analyses. However, this 40-item scale is too time demanding. Therefore, in order to reduce the time required for its administration, Lim and Chapman (2013) developed a shortened version. The confirmatory analyses yielded sound properties, but a high correlation coefficient was found between the enjoyment and motivation dimensions (r = .96). This result indicated that these two latent factors were statistically isomorphic and therefore, a reduction of the factor structure to three factors would presumably yield a better model fit to the data. On the other hand, some instruments, although primarily developed to measure attitudes towards mathematics, actually comprise in the same scale both attitudinal dimensions (e.g., motivation, perceived usefulness) and mathematics anxiety. That is the case of the FennemaSherman Mathematics Attitude Scales (FSMAS; Fennema & Sherman, 1976), Mathematics Attitude Inventory (MAI; Sandman, 1980), Escala de Actitudes hacia las Matemáticas (EAM; Auzmendi, 1992), Escala de Actitudes hacia la Matemática-Universidad (EAHM-U; Bazán, 1997) and Short form of Mathematics Attitude Scale (Yasar, 2014). Nevertheless, as Evans (2006) claimed, attitudes towards mathematics and mathematics anxiety are two separate subdomains of the more general domain mathematical affect. This means that the construct attitudes towards mathematics has its own factor structure and its assessments should be tested separately from mathematics anxiety. Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 562 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 Moreover, research on the subdomain attitudes towards mathematics has extensively acknowledged its multidimensional nature and has identified student’s confidence as a salient underlying variable (Ruffell, Mason, & Allen, 1998; Gómez-Chacón, 2000; Hanulla, 2002; Di Martino & Zan, 2010). In some cases, student’s confidence is not included, such as in the Math Attitude Scale (Aiken & Dreger, 1961), the Dutton Scale (DAS; Dutton & Blum, 1968), Enjoyment and Value scales (E and V scales; Aiken, 1974), Instrument measuring certain attitudes toward mathematics (Michaels & Forsyth, 1977) and Cuestionario para medir las actitudes hacia las matemáticas en alumnos de ESO (Muñoz & Mato, 2006). In other cases, this variable appears divided into two subfactors, such as in the scale for measuring attitudes toward mathematics in Compulsory Secondary Education (Alemany-Arrebola & Lara, 2010) and Escala de Actitudes hacia las Matemáticas (EAM; Palacios, Arias, & Arias, 2014). In the former, the authors distinguished between positive and negative self-concept; in the latter, the authors distinguished between the perceived mathematical incompetence and self-concept. However, due to their nature, in both cases, the two subfactors considered by the authors should be constituent of the same homogeneous dimension. The current study Drawing on the literature review, the comprehensive analysis of the content of Table 1 suggests the item redistribution in three main categories: student’s math self-concept, perceived usefulness of mathematics and interest for mathematics. Table 1. Measurements for assessing attitudes towards mathematics Instrument Dimensions/Items Psychometric evidence Math Attitude Scale (Aiken & Dreger, 1961) A unidimensional 20-item scale consisted of 10 items connoting negative feelings and 10 items connoting positive feelings. EFA Test-retest reliability: .94 (N=127) The Dutton Scale (DAS; Dutton & Blum, 1968) A homogeneous scale consisting of 27 items that discriminate between positive and negative feelings about arithmetic. Spearman-Brown reliability for the full scale: .84 (N=346) Enjoyment and Value Scales (E scale and V scale; Aiken, 1974) A set of two scales, which can be used either separately or jointly: Enjoyment scale (11) and Value scale (10) EFA Cronbach’s alpha for E scale: .95 Cronbach’s alpha for V scale: .85 (N = 190) Fennema-Sherman Mathematics Attitude A set of nine subscales, which can be used either separately or jointly: Mathematics anxiety scale (12), attitude toward EFA Split-half reliability Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 563 - http://dx.doi.org/10.14204/ejrep.40.15163 Scales (FSMAS; Fennema & Sherman, 1976) success in Mathematics scale (12), confidence in learning Mathematics scale (12), effectance motivation in Mathematics scale (12), father scale (12), mother scale (12), Mathematics as a male domain (12), teacher scale (12), usefulness of Mathematics scale (12). (for the subscales): .86-.93 (N=1,600) Instrument measuring certain attitudes toward mathematics (Michaels & Forsyth, 1977) A set of four subscales: enjoyment of word problems (1), enjoyment of pictorial problems (1), appreciation of the utility of mathematics (10), security with mathematics (10), EFA Spearman-Brown reliability (for the subscales): .51-.78 (N=299) Mathematics Attitude Inventory (MAI; Sandman, 1980) A set of six subscales, resulting in a total of 48 items: perception of Mathematics teachers, value of Mathematics, self-concept in Mathematics, math anxiety, enjoyment of Mathematics, motivation in Mathematics. EFA Reliability (for the subscales): .69-.89 (N=5,034) Escala de actitud de carácter verbal (Gairín, 1990) A 22-item scale with three underlying factors: liking, usefulness, and confidence-anxiety. EFA Test-retest reliability (for the dimensions): .77-.93 (N=3,637) Escala de Actitudes hacia las Matemáticas (EAM; Auzmendi, 1992) A 25-item scale with five underlying factors: usefulness (5), confidence (5), anxiety (5), liking (5), motivation (5). EFA (PCA and VR) Cronbach’s alpha (for the dimensions): .50-.91 Cronbach’s alpha for the full scale: .93 (N=1,221) Escala de Actitudes hacia la MatemáticaUniversidad (EAHM-U; Bazán, 1997) A 31-item scale with four underlying factors: affectivity (8), applicability (8), ability (8) and anxiety (7). EFA Cronbach’s alpha (for the dimensions): .71-.91 Attitudes toward Mathematics and Mathematics Taught with Computer (AMMEC; Ursini, Sánchez, & Orendain, 2004) A 29-item scale comprising three subscales: liking for mathematics (11), liking for mathematics taught with computer (11), self-confidence (7). EFA (PCA and VR) Cronbach’s alpha (for the subscales): .68-.81 Cronbach’s alpha for the full scale: .80 Split-half reliability for the full scale: .71 (N=439) Attitudes toward Mathematics Inventory (ATMI; Tapia & Marsh, 2004) A 40-item scale with four underlying dimensions: selfconfidence (15), value (10), enjoyment (10), motivation (5) EFA (ML and VR) and CFA Cronbach’s alpha (for the dimensions): .88-.95 Cronbach’s alpha for the full scale: .97 Test-retest reliability (for the dimensions): .70-.80 Test-retest reliability for the full scale: .89 (N=545) Cuestionario para medir las actitudes hacia las matemáticas en alumnos A 19-item scale with two underlying factors: liking and usefulness (9), teacher’s attitude toward mathematics perceived by the student (10) EFA (PCA and VR) Cronbach’s alpha for the full scale: .97 Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 564 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 de ESO (Muñoz & Mato, 2006) (N=1,220) Scale for measuring attitudes toward mathematics in compulsory secondary education (Alemany-Arrebola & Lara, 2010) A 35-item scale with seven underlying factors: behavioural component (13), affective component (7), negative selfconcept (5), positive self-concept (3), cognitive component (3), demotivation towards mathematics (2), expectancy (2) EFA and CFA Cronbach’s alpha (for the dimensions): .43-.89 Cronbach’s alpha for the full scale: .92 (N=236) Shortened version of the Attitudes toward Mathematics Inventory (short ATMI; Lim & Chapman, 2013) A 19-item shortened ATMI version with four subscales: enjoyment (5), motivation (4), self-confidence (5), perceived value (5). EFA and CFA Cronbach’s alpha (for the subscales): .85-.90 Cronbach’s alpha for the full scale:.93 Test-retest reliability for the full scale: .75 (N=1,601) Short form of Mathematics Attitude Scale (Yasar, 2014) A 19-item scale with four underlying dimensions: enjoyment (6), fear, anxiety and boredom (5), place of mathematics in life (4), perceived mathematics success (4). EFA (BCA) and CFA Cronbach’s alpha (for the dimensions): .82-.89 Cronbach’s alpha for the full scale: .96 (N=1,801) Escala de Actitudes hacia las Matemáticas (EAM; Palacios, Arias, & Arias, 2014) A 32-item scale underlying four dimensions: perception of mathematical incompetence (12), liking (12), perception of usefulness (4), mathematical self-concept (4) EFA (PAF, ML and PR) and CFA Cronbach’s alpha (for the dimensions):.68-.93 Cronbach’s alpha for the full scale: .95 (N=4,807) Note. BCA=Basic Component Analysis, CFA=Confirmatory Factor Analysis, EFA=Exploratory Factor Analysis, ML=Maximum Likelihood, PCA=Principal Component Analysis, PR=Promax Rotation, VR=Varimax Rotation. Using this categorization as a basis for the present study, the theoretical model proposed to be tested is tri-dimensional and non-hierarchized, with the following definitions for the three related first-order latent factors: Student’s math self-concept: encompasses a broad range of student’s responses about her or his ability to learn and do mathematics (e.g., “I am unable to solve math problems”). A student who scores high in this dimension believes that she or he has the ability to understand and solve math-related tasks. On the other hand, a student scoring low in this dimension does not believe that she or he has the ability to understand and do mathematics. Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 565 - http://dx.doi.org/10.14204/ejrep.40.15163 Perceived usefulness of mathematics: measures the students’ extrinsic utility value of mathematics. This dimension, as defined by Eccles and Wigfield (2002), measures students’ beliefs about the applicability of mathematics for their current and future goals and in relation to school, career and everyday life (e.g., “Learning math will increase my future job opportunities”). Therefore, a student who scores high in this dimension finds mathematics very useful for both their current and future goals; whereas a student scoring low finds it useless for both their current and future goals. Interest for mathematics: refers to the amount of interest students have in learning and doing mathematics (e.g., “Time just flies by when I am solving math problems”). A student who scores high in this dimension has high interest in learning and doing mathematics. On the other hand, a student scoring low dislikes mathematics, finds it boring and does not take pleasure from learning and doing mathematics. Method Participants The sample consisted of 792 secondary students, which were selected via a clustersampling method from 36 classes from the province of Biscay (Basque Country Autonomous Region, Spain). This sample was then divided into two subgroups according to the language in which they learn mathematics (see Table 2). Table 2. Sociodemographic profile of the sample 2nd grade 4th grade Total Females Males Females Males Subsample 1 45 72 88 138 343 Subsample 2 137 138 88 86 449 Total 182 210 176 224 792 Instruments Scale for assessing attitudes towards mathematics in secondary education (SATMAS) A pool of items was collected from all the examined instruments (see Table 1) and redistributed in the aforementioned three dimensions. After removing redundant items, rewording some others and adding a few newly written ones, a final pool of 36 items was obtained Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 572 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 the best structure to explain the construct attitudes toward mathematics, as it showed a simpler structure compared to that of M1a. Table 6. Goodness-of-fit indices for the revised structures Model Subsample χ 2 S-B /df NNFI CFI IFI SMSR RMSEA (90% CI) AIC M1 S1 3.56 .90 .91 .91 .071 .068 [.062, .072] 354.52 S2 2.06 .90 .91 .91 .079 .068 [.059, .077] 13.683 M1a S1 2.78 .94 .95 .95 .053 .056 [.050, .062] 128.803 S2 1.62 .95 .96 .96 .058 .052 [.040, .063] -63.335 M1b S1 3.28 .93 .94 .94 .056 .064 [.057, .070] 190.994 S2 1.77 .94 .95 .95 .060 .058 [.046, .069] -34.504 M1c S1 2.88 .94 .95 .95 .053 .058 [.051, .064] 130.788 S2 1.54 .96 .96 .96 .057 .049 [.036, .061] -68.206 Final model In addition, all standardized factor loadings and inter-factor correlations were statistically significant (p < .05), with values ranging from .49 to .92 for factor loadings and from .21 to .73 for inter-factor correlations (see Figure 1). These results underscored the discriminant validity of the scale between the three underlying latent factors. Figure 1. Confirmatory factor analysis of SATMAS with subsample 1 (N = 563) and subsample 2 (N = 229) Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 573 - http://dx.doi.org/10.14204/ejrep.40.15163 Note. MSC=Math Self-Concept, PUM=Perceived Usefulness of Mathematics, IM=Interest for Mathematics. Additional properties of the last 19-item SATMAS were assessed with the composite reliability (CR) and Cronbach’s coefficient ( α ) of each dimension. The reliability analyses showed good internal consistency of the student’s math self-concept (CR s1 = .93, α s1 = .93; CR s2 = .89, α s2 = .89), perceived usefulness of mathematics (CR s1 = .78, α s1 = .78; CR s2 = .80, α s2 = .80) and interest for mathematics (CR s1 = .90, α s1 = .91; CR s2 = .90, α s2 = .91). Regarding the full scales, reliabilities were α 1 = .93 and α 2 = .91 for Subsample 1 and Subsample 2, respectively. The reliability scores were found to be good based on the Nunnally’s criterion (Nunnally, 1978). Therefore, the results suggested that the items were internally consistent in representing the corresponding factor. Finally, criterion-related validity for the scale was assessed by the Pearson correlation coefficients between attitudes towards mathematics and the scores obtained in mathematics achievement tests. Positive significant correlations were found, as expected, between student’s math self-concept and math achievement (r s1 = .23, p < .001; r s2 = .22, p < .001), perceived usefulness of mathematics and math achievement (r s1 = .22, p < .001; r s2 = .23, p < .001) and interest for mathematics and math achievement (r s1 = .20, p < .001; r s2 = .21, p < .001). The full scale is included in Table 7. Table 7. Last version of the 19-item SATMAS Student’s math self-concept IT01 I feel more foolish than my classmates when solving math problems and exercises IT02 In spite of my effort, I cannot understand math IT03 I have difficulties with math IT04 I was not a born math learner IT05 I am unable to solve math problems IT06 Whatever I do, I get low grades in math IT07 It will be always hard for me to learn math Perceived usefulness of mathematics IT08 Math is very useful IT09 Everybody needs to learn math IT12 Math is necessary for life IT13 Math is important for society development IT15 Learning math is important for my future job Interest for mathematics IT17 I like studying math IT18 I like math IT19 Time just flies by when I am studying math Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 574 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 IT20 Studying math is fun IT21 Time just flies by when I am solving math problems / exercises IT22 Math is entertaining IT23 Math is a drag Discussion Over the past years, there has been a growing interest in studying the students’ attitudes towards mathematics because of their important role in the engagement in and mastery of mathematics (e.g., McLeod, 1992; Goldin, 2002; Grootenboer & Hemmings, 2007; Malmivouri, 2007). Nevertheless, correlational research on attitudinal variables and mathematical performance has been contingent on the psychometric properties of the measurements for measuring the construct attitudes towards mathematics. Three conclusions may be drawn from a comprehensive literature review on the existing instruments. Firstly, the most widely cited instruments are the FSMAS (Fennema & Sherman, 1976) and the ATMI (Tapia & Marsh, 2004), which have also been translated into several languages for their use in backgrounds with different socio-cultural characteristics. However, subsequent replication studies of these instruments (e.g., O’Neal, Ernest, McLean, & Templeton, 1988; Melancon, Thompson, & Becnel, 1994; Mulhern & Race, 1998) have obtained evidence that rebuilding some of their latent factors and shortening the scales to fewer subdomains would yield a better fit to data. Secondly, there are some instruments (i.e., MAI, Sandman, 1980; EAM, Auzmendi, 1992; EAHM-U, Bazán, 1997; Short Form of Mathematics Attitude Scale, Yasar, 2014) that, although primarily set to measure attitudes towards mathematics, actually mix both attitudinal factors and mathematics anxiety. Nevertheless, attitudes towards mathematics and mathematics anxiety are claimed to be considered as two separate subdomains of the more general domain mathematical affect (Evans, 2006). This means that attitudes towards mathematics has its own factor structure and its assessments should be tested separately from mathematics anxiety. Thirdly, they are other measurements that do not include student’s self-confidence as an underlying factor (i.e., in the Math Attitude Scale, Aiken & Dreger, 1961; DAS, Dutton & Blum, 1968; E and V Scales, Aiken, 1974; Instrument measuring certain attitudes toward mathematics, Michaels & Forsyth, 1977; and Cuestionario para medir las actitudes hacia las matemáticas en alumnos de ESO, Muñoz & Mato, 2006) or that include it in such a way that remains ambiguous (i.e., in the scale for measuring attitudes towards mathematics in Compul- Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 575 - http://dx.doi.org/10.14204/ejrep.40.15163 sory Secondary Education, Alemany-Arrebola & Lara, 2010; and EAM, Palacios, Arias, & Arias, 2014). As a result of this lack of consistency in defining the theoretical conceptualization of the construct attitudes towards mathematics, this paper aims to develop and validate, by means of confirmatory techniques, a structural model for this construct. Drawing on the comprehensive analysis of the existing instruments, three latent factors are tested as first-order constituents of the attitudes towards mathematics, considering this as a non-hierarchized structure: a) students’ math self-concept, b) perceived usefulness of mathematics, and c) interest for mathematics. The sample, consisting of 792 compulsory secondary students, was divided into two subgroups according to the language in which they learned mathematics (Spanish or Basque). Goodness-of-fit indices were calculated for the theoretical model and further improvements were made after examining standardized factor loadings, modification covariance indices and standardized residual covariance scores. The last version, obtained after eliminating four original items and forcing to covariate to another two pairs of items, yielded a better model fit. The goodness-of-fit indices were found to be very similar in both student subgroups. Internal consistency, measured by Cronbach’s coefficient, had an average value of .92, which was referred to as excellent according to Nunally’s criterion (Nunnally, 1978). Discriminant and criterion-related validities were also assessed. On the one hand, inter-factor correlation scores were positive and ranged from .21 to .73, meaning that the latent factors were not statistically isomorphic. On the other hand, bivariate correlation analyses yielded significant positive correlation scores between each attitudinal dimension and the score obtained in the mathematical achievement test, developed ad-hoc. Although statistically significant, these correlation scores were found to be small, which was explained by the effect that other variables, not considered in this paper, had on students’ mathematical achievement. In fact, a literature review draws attention to the fact that mathematics achievement is affected not only by cognitive and attitudinal factors, but also by school environment (Creemers & Reezigt, 2006) or instructional strategies (Van de Grift & Houtveen, 2006). Despite the promising findings, there are also some methodological limitations that warrant cautious considerations in generalizing the results. First, data were collected solely from Biscay (Basque Country Autonomous Region, Spain), meaning that the results are not entirely generalizable outside of a Basque population. Nevertheless, there are some pieces of evidence suggesting that it is possible to rely on the quality of the data. On the one hand, the student Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 576 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 sample was selected via a cluster-sampling method, and the descriptive statistics of the resulting group were very similar to that of the reference population, according to the official enrolment data of the Education Department of Basque Government for the school year in which the study was carried out. As a result, the sample group was representative and large enough for the research purposes, although not completely probabilistic in case of generalizing the findings outside Biscay. In this line, future research using larger samples from different sociodemographic contexts would be necessary to further assess the invariance of the factor structure. On the other hand, this study provides strong reliability and validity evidence of the instrument (i.e., construct validity, discriminant validity, criterion-related validity and internal consistency), but it would be interesting to assess the structural stability through testretest reliability analyses. Given the reliability and validity evidence gathered in the present study, the 19-item SATMAS proves to be a promising instrument for assessing secondary students’ attitudes towards mathematics. On the one hand, the results largely supported the theoretical conceptualization according to which attitudes towards mathematics is a multidimensional construct with a non-hierarchized structure consisting of the three aforementioned first-order factors (namely, student’s math self-concept, perceived usefulness of mathematics and interest for mathematics). This was found to be a great contribution to the research on attitudes towards mathematics. On the other hand, the developed scale is easy to administer and not timedemanding, as this short form takes secondary students 15 minutes to complete. Therefore, either school counselors or educators might use it to measure students’ attitudes towards mathematics and provide early attention measures in case the levels of mathematical selfconcept and motivation were low. This might be of great interest particularly for those students showing strong mathematical skills but struggling with low expectations and interest, which put them at risk of disengagement in mathematics and mathematics-related pathways. In fact, since attitudes towards the subject have been shown to decline from upper elementary school to junior high school, developing and validating a measurement targeting secondary students would furnish insights within the field of mathematics education and may well become a starting point to identify and prevent those dropout risky situations at mathematics classrooms. On other hand, researchers might use it as the starting point to identify the key domains of attitudes towards mathematics affecting the mathematical achievement and further investigate the variables which affect their prevalence. In fact, assessing the dimensions un- Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 577 - http://dx.doi.org/10.14204/ejrep.40.15163 derlying attitudes towards mathematics is critical to gain knowledge about the plausible factors that affect mathematical achievement. Acknowledgements This research was funded by the University of Deusto Research Training Grants Programme 2013-2016 (University of Deusto, Bilbao). References Aiken, L.R. (1974). Two scales of attitude toward mathematics. Journal for Research in Mathematics Education, 5, 67-71. doi: 10.2307/748616 Aiken, L.R., & Dreger, R.M. (1961). The effect of attitudes on performance in learning mathematics. Journal of Educational Psychology, 52(1), 19-24. Alemany-Arrebola, I., & Lara, A.I. (2010). Las actitudes hacia las matemáticas en el alumnado de ESO: un instrumento para su medición. Publicaciones, 40, 49-71. Anjum, R. (2006). The role of self-concept in mathematics achievement of primary school children. Pakistan Journal of Social and Clinical Psychology, 4, 21-38. Auzmendi, E. (1992). Las actitudes hacia la matemática-estadística en las enseñanzas medias y universitaria: Características y medición. Bilbao: Mensajero. Barth, J.M., Todd, B., McCallum, D.M., Goldston, M., Guadagno, R.E., Roskos, B., & Burkhalter, C. (2011). Effects of engaging classroom strategies and teacher support on student outcomes over school transitions. Proceedings of the American Society for Engineering Education. Bazán, J.L. (1997). Metodología estadística de construcción de pruebas. Una aplicación al estudio de actitudes hacia las matemáticas en la UNALM. UNALM: Lima (tesis). Bentler, P.M. (2005). EQS Structural equations program manual. Encino, CA: Multivariate Software. Inc. Bouchey, H.A., & Harter, S. (2005). Reflected appraisals, academic perceptions, and math/science performance during early adolescence. Journal of Educational Psychology, 97(4), 673-686. doi: 10.1037/0022-0663.97.4.673 Brown, T. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press. Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 578 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 Creemers, B.P.M., & Reezigt, G.J. (2005). Linking school effectiveness and school improvement: The background and outline of the project. School effectiveness and school improvement, 16(4), 359-371. doi: 10.1080/09243450500234484 Crombie, G., Sinclair, N., Silverthorn, N., Byrne, B.M., Dubois, D.I., & Trinneer, A. (2005). Predictors of young adolescents’ math grades and course enrollment intentions: Gender similarities and differences. Sex Roles, 52(5-6), 351-367. doi: 10.1007/s11199005-2678-1 Di Martino, P., & Zan, R. (2010). ‘Me and maths’: Towards a definition of attitude grounded on students’ narratives. Journal of Mathematics Teacher Education, 13, 27-48. doi: 10.1007/s10857-009-9134-z Dutton, W.H., & Blum, M.P. (1968). The measurement of attitudes toward arithmetic with a Likert-type test. Elementary School Journal, 68, 259-264. doi: 10.1086/460443 Eccles, J.S., & Wigfield, A. (2002).Motivational beliefs, values, and goals.Annual Review of Psychology, 53, 109-132. doi: 10.1146/annurev.psych.53.100901.135153 Fennema, E., & Sherman, J.A. (1976). Fennema-Sherman Mathematics Attitudes Scales: Instruments designed to measure attitudes toward the learning of mathematics by females and males. Journal for Research in Mathematics Education, 7, 324-336. doi:10.2307/748467 Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. doi: 10.2307/3151312 Gairín, J. (1990). Las actitudes en educación. Un estudio sobre la educación matemática. Barcelona: Boixareu Universitaria. Goldin, G. (2002). Affect, meta-affect, and mathematical belief structures. In G. Leder, E. Pehkonen & G. Törner (Eds.). Beliefs: A hidden variable in mathematics education? Dordrecht: Kluwer. Gómez-Chacón, I.M. (2000). Affective influences in the knowledge of mathematics. Educational Studies in Mathematics, 43(2), 149-168. doi: 10.1023/A:1017518812079 Grootenboer, P., & Hemmings, B. (2007). Mathematics performance and the role played by affective and background factors. Mathematics Education Research Journal, 19(3), 310. doi: 10.1007/BF03217459 Hair, J.F., Black, W.C., Babib, B.J., & Anderson, R.E. (2010). Multivariate data analysis (7 th ed.). Upper Saddke River, NJ: Prentice Hall. Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 579 - http://dx.doi.org/10.14204/ejrep.40.15163 Helmke, A. (1989). Incentive value of success and failure in school. Developmental trends and impact on academic achievement. In F. Halisch & J.J.L. Van den Berchen (Eds.), International perspectives on achievement and task motivation (225-237). Lisse: Swets & Zeitlinger. Hoyle, R.H., & Panter, A.T. (1995). Writing about structural equation models in structural equation modeling. In R.H. Hoyle (Ed.). Structural Equation Modeling: Concepts. Issues and Applications (158-176). Thousand Oaks, CA: Sage. Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Quotation Modeling, 6, 1-55. doi: 10.1080/10705519909540118 Ignacio, N.T., Nieto, L.J., & Barona, E.G. (2006). The affective domain in mathematics learning. International Electronic Journal of Mathematics Education, 1(1), 16-32. Kadijevich, D. (2008). TIMSS 2003: Relating dimensions of mathematics attitude to mathematics achievement (MA). Zbornik Instituta za pedagoska istrazivanja, 40(2), 327346. doi: 10.2298/ZIPI0802327K Lei, M., & Lomax, R.G. (2005). The effect of varying degrees of nonnormality in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 12, 127. doi: 10.1207/s15328007sem1201_1 Lim, S.Y., & Chapman, E. (2013). Development of a short form of the attitudes toward mathematics inventory. Educational Studies in Mathematics, 83, 145-164. doi: 10.1007/s10649-012-9414-x Lipnevich, A.A., MacCann, C., Krumm, S., Burrus, J., & Roberts, R.D. (2011). Mathematics attitudes and mathematics outcomes of U.S. and Belarusian Middle School Students. Journal of Educational Psychology, 103(1), 105-118. doi: 10.1037/a0021949 Ma, X., & Kishor, N. (1997). Assessing the relationship between attitudes towards mathematics and achievement in mathematics: A meta-analysis. Journal for Research in Mathematics Education, 28(1), 26-47. doi: 10.2307/749662 Mahoney, C.A., Thombs, D.L., & Howe, C.Z. (1995). The art and science of scale development in health education research. Health Education Research,10, 1-10. doi: 10.1093/her/10.1.1-a McLeod, D.B. (1992). Research on affect in mathematics education: A reconceptualisation. In D.A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (575-596). New York, NY: McMillan. Lara Yáñez-Marquina & Lourdes Villardón-Gallego - 580 - Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 http://dx.doi.org/10.14204/ejrep.40.15163 Melancon, J.G., Thomson, B., & Becnel, S. (1994). Measurement integrity of scores from the Fennema-Sherman Mathematics Attitudes Scales: The attitudes of public school teachers. Educational and Psychological Measurement, 54, 187-192. doi: 10.1177/0013164494054001024 Michaels, L.A., & Forsyth, R.A. (1977). Construction and validation of an instrument measuring certain attitudes toward mathematics. Educational and Psychological Measurement, 37(4), 1043-1049. doi: 10.1177/001316447703700429 Mulhern, F., & Race, G. (1998). Development of a shortened form of the Fennema-Sherman Mathematics Attitudes Scales. Educational and Psychological Measurement, 58, 295306. doi: 10.1177/0013164498058002012 Muñoz, J.M., & Mato, M.D. (2006). Diseño y validación de un cuestionario para medir las actitudes hacia las matemáticas en alumnos de ESO. Revista galego-portuguesa de psicoloxía e educación, 13, 413-424. Nunnally, J.C. (1978). Psychometric theory (2 nd ed.). New York, NY: McGraw-Hill. O’Neal, M.R., Ernest, P.S., McLean, J.E., & Templeton, S.M. (1988, November). Factorial validity of the Fennema-Sherman Mathematics Attitude Scales. Paper presented at the annual meeting of the Mid-South Educational Research Association, Louisville, KY. Palacios, A., Arias, V., & Arias, B. (2014). Attitudes Towards Mathematics: Construction and Validation of a Measurement Instrument. Journal of Psicodidactics, 19(1), 67-91. doi: 10.1387/RevPsicodidact.8961 Ruffell, M., Mason, J., & Allen, B. (1988). Studying attitude to mathematics. Educational Studies in Mathematics, 35, 1-18. Samuelsson, J., & Granstom, K. (2007). Important prerequisite for students’ mathematical achievement. Journal of Theory and Practice in Education, 3(2), 150-170. Sandman, R.S. (1980). The mathematics attitude inventory: Instrument and user’s manual. Journal for Research in Mathematics Education, 11(2), 148-149. doi: 10.2307/748906 Satorra, A. (2003). Power of chi-square Goodness-of-fit test in structural equation models: the case of non-normal data. In H. Yanai, A. Okada, K. Shigemasu, Y. Kano & J. J. Meulman (Eds.). New developments of psychometrics (57-68). Tokio: Springer Verlag. Tabachnick, B.G., & Fidell, L.S. (2007). Using multivariate statistics (5 th ed.). Boston, MA: Allyn & Bacon. Tapia, M. & Marsh, G. E. (2004). An Instrument to Measure Mathematics Attitudes. Academic Exchange Quarterly, 8(2), 16-21. Attitudes towards mathematics at secondary level: Development and structural validation of the Scale for Assessing Attitudes towards Mathematics in Secondary Education (SATMAS) Electronic Journal of Research in Educational Psychology, 14(3), 557-581. ISSN: 1696-2095. 2016. no. 40 - 581 - http://dx.doi.org/10.14204/ejrep.40.15163 Ullman, J.B. (2006). Structural equation modeling. In B.G. Tabachnick & L.S. Fidell (Eds.). Using multivariate statistics (5 th ed), (653-771). Boston, MA: Allyn & Bacon. Ursini, S., Sánchez, G., & Orendain, M. (2004). Validación y confiabilidad de una escala de Actitudes hacia las Matemáticas y hacia las Matemáticas Enseñadas con Computadora. Educación Matemática, 16(3), 59-78. Van de Grift, W., & Houtveen, T. (2006). Underperformance in primary schools. School Effectiveness and School Improvement, 17, 1-19. doi: 10.1080/09243450600697317 Watt, H.M.G. (2000). Measuring attitudinal change in mathematics and English over the first year of junior high school: A multidimensional analysis. Journal of Experimental Education, 68, 331-361. doi: 10.1080/00220970009600642 Williams, T., & Williams, K. (2010). Self-efficacy and performance in mathematics: Reciprocal determinism in 33 nations. Journal of Educational Psychology, 102, 453-466. doi: 10.1037/a0017271 Yasar, M. (2014). Short form of Mathematics Attitude Scale: Its psychometric properties. Pakistan Journal of Statistics, 30(6), 1267-1277 Zan, R., Brown, L., Evans, J., & Hannula, M.S. (2006). Affect in mathematics education: An introduction. Educational Studies in Mathematics, 63(2), 113121.doi: 10.1007/s10649-006-9028-2