The determinants of mathematics achievement: A gender perspective using multilevel random forest

Bertoletti, Alice et al. Article The determinants of mathematics achievement: A gender perspective using multilevel random forest Economies Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Bertoletti, Alice et al. (2023) : The determinants of mathematics achievement: A gender perspective using multilevel random forest, Economies, ISSN 2227-7099, MDPI, Basel, Vol. 11, Iss. 2, pp. 1-20, https://doi.org/10.3390/economies11020032 This Version is available at: https://hdl.handle.net/10419/328658 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Citation: Bertoletti, Alice, Marta Cannistrà, Melisa Diaz Lema, Chiara Masci, Anna Mergoni, Lidia Rossi, and Mara Soncin. 2023. The Determinants of Mathematics Achievement: A Gender Perspective Using Multilevel Random Forest. Economies 11: 32. https://doi.org/ 10.3390/economies11020032 Academic Editors: Giorgio Vittadini, Tommaso Agasisti, Roberto Ricci and Lanfranco Senn Received: 29 November 2022 Revised: 5 January 2023 Accepted: 11 January 2023 Published: 17 January 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). economies Article The Determinants of Mathematics Achievement: A Gender Perspective Using Multilevel Random Forest Alice Bertoletti 1,†,‡ , Marta Cannistrà 2,‡ , Melisa Diaz Lema 2,‡ , Chiara Masci 3,‡ , Anna Mergoni 4,‡ , Lidia Rossi 2,‡ and Mara Soncin 2,*,‡ 1Joint Research Center, European Commission, 41092 Seville, Spain 2 Department of Management, Economics and Industrial Engineering, Politecnico di Milano, 20133 Milan, Italy 3Department of Mathematical Engineering, Politecnico di Milano, 20133 Milan, Italy 4Faculty of Economics and Business, KU Leuven, 3000 Leuven, Belgium *Correspondence: [email protected] † The views expressed are purely those of the author and may not in any circumstances be regarded as stating an official position of the European Commission. ‡ These authors contributed equally to this work. Abstract: This paper investigates the determinants of mathematics performance by gender, exploiting a multilevel random forest approach. OECD PISA 2018 data from 28 European countries are employed to explore the performance of male and female students as a function of students’ family characteristics, their attitudes towards education, and class and school environment. Results show that the gender gap in favour of boys persists in most European countries. However, teacher and school practices like fostering student reading and creating a cooperative environment allow mitigating the influence of family background in countries without gender gap. Policy implications to foster performance equality are provided. Keywords: mathematics achievement; comparative analysis; gender gap; random forest 1. Introduction and Motivation Equality of opportunity across individuals is a matter of primary importance in the political agenda of worldwide economies (Dunnzlaff et al. 2011). To reach this objective, a fair educational system is a necessary step, given that higher educational levels are associated with higher wages, better health, and higher well-being level. In particular, gender inequality represents an unresolved question that ranges from reduced women’s participation in the labour market to salary gaps and gender stereotyping in career choice (Education et al. 2012). It is particularly evident how women are structurally underrepresented in science, technology, engineering, and math (STEM) careers, making this an untapped opportunity to expand employability and innovation capacity (Beede et al. 2011). Gender studies have traditionally traced back these gender differences to disparities in educational outcomes (Evans et al. 2020). While girls tend to outperform boys in reading (van Hek et al. 2019), the gap in mathematics is structurally in favour of boys in most European countries (Contini et al. 2017;Education et al. 2012). Despite the net difference in mathematics being usually smaller than the gap in reading, the amplification effects in terms of different career choices and salary gap in favour of men are relevant enough to make this a question of primary relevance (UNICEF 2020). Lower math achievement leads, in many cases, to lower participation of females in STEM majors at university (Card and Payne 2021). In turn, this easily translates into gender gaps in the labour market and occupational choices in disfavour of women (Bertocchi and Bozzano 2020;Machin and Puhani 2003; Piazzalunga 2018). The paper addresses this issue by focusing on student achievement in mathematics and investigating the causes leading to disparities across gender. Economies 2023,11, 32. https://doi.org/10.3390/economies11020032 https://www.mdpi.com/journal/economies Economies 2023,11, 32 2 of 20 Extant studies tend to explain the gender gap in education by looking at differences in the level of the main determinants of school achievement between boys and girls (Figlio et al. 2019;Buchmann et al. 2008) . However, the decomposition analyses available in the literature generally reveal that differences in the key factors predicting learning achievement (such as household resources, parents’ and teachers’ support, family expectations, and career motivation) can only partially explain the gender gap in education (Gevrek et al. 2020;Munir and Winter-Ebmer 2018). While cultural and societal dimensions can play a relevant role (Else-Quest et al. 2010;Giuliano 2020), unexplained educational differences between males and females may also be associated with structural differences in the way in which some key factors influence student performance across gender. Thus, the present paper relaxes the baseline assumption that the determinants of educational achievement have the same impact across gender by modelling the determinants of boys’ and girls’ performance separately. Moreover, the study adopts an international approach, by exploring the European countries as empirical context. In particular, the research addresses the following question: How do the determinants of mathematics achievement differ between male and female students and among European countries? The paper explores the OECD Programme for International Student Assessment (PISA) 2018 dataset for 28 European countries (i.e., the EU countries with available data, plus the UK, Iceland, and Switzerland). The data provided by PISA refers to 15-year-old students and, therefore, can be employed to examine the gender gap in a crucial moment of education, corresponding to the last year of compulsory school in most countries. To ensure homogeneity in structural characteristics, the analyses are carried out by classifying countries into three categories, i.e., the ones with a gender gap in favour of boys, the ones with a gender gap in favour of girls, and the ones with no gap. A multilevel random forest (Pellagatti et al. 2021), where student and country levels are considered, is implemented separately in the three groups of countries. We follow a random forest approach because its flexibility adapts well to the educational context, in which several input variables co-exist in the same environment (Masci et al. 2018). More specifically, while more classic linear multilevel models are able to estimate only linear associations between covariates and the response, this technique, by relaxing any a priori parametric assumption, performs well in presence of several interactions among predictors and allows to discover the most likely relationship between the variables. At the same time, the multilevel approach allows modelling the heterogeneity between countries and to disentangle the variability given at student and country levels. The empirical results indicate the existence of structural differences in some relevant determinants of math achievement between boys and girls, such as perception of cooperation and reading attitudes. The way through which factors influence math performances of males and females is also strongly related to the geographic area in which pupils are studying. To this extent, the paper presents useful evidence to design specific policy actions for enhancing gender equality in education and the labour market. The remainder of the paper is organised as follows. Section 2revises the literature on the determinants of the gender gap in mathematics and presents the conceptual framework. Section 3describes the data and the methodology used for the empirical analyses. Then, the results and their discussion are presented in Sections 4and 5, while final implications and conclusions are reported in Section 6. 2. Literature Review 2.1. Evidence of Gender Gap in Student Performance The existence and persistence of a gender gap in mathematics in favour of boys has been demonstrated by multiple studies over time (Borgonovi et al. 2018;Contini et al. 2017;Frye and Levitt 2010;González de San Román and De La Rica 2012). What is particularly striking from the current literature is the absence of a gap between boys and girls when they enter school, while it becomes larger with the years of schooling (Borgonovi et al. 2018;Mejias et al. 2021). Frye and Levitt (2010) show that the gap in mathematics increases from 0 to 0.2 standard Economies 2023,11, 32 3 of 20 deviations after 6 years of education. This gap becomes particularly pronounced after males and females leave compulsory schooling and enter in post-compulsory education and the labour market, with important effects on students’ educational trajectories and opportunities. Multiple possible explanations have been attempted, ranging from less involvement in maths for girls to low parental expectations, but the determinants of such a phenomenon are still highly debated (Bouffard and Hill 2005;Frye and Levitt 2010;Levine et al. 2005). The Programme for International Student Assessment (PISA), an international survey of 15-year-old students among OECD countries, has often been employed to study the extent of the gender gap internationally. Gender gap in mathematics performance remained broadly stable between PISA 2012 and PISA 2015 (OECD 2016), showing, if anything, a small reduction of boys’ advantage in mathematics. In 2012, boys outperformed girls in mathematics in 38 of the 65 participating countries by an average of 11 score points (across OECD countries) (OECD 2019a), while in 2018 boys significantly outperformed girls in 32 of the 79 participating countries by an average of 6 points. Interestingly, in 2018, 14 countries showed an opposite gender gap in mathematics (OECD 2019a). Among these economies, Finland represents the European country where girls obtained the highest scores with respect to boys in mathematics, on average. On the opposite, in 2018, the largest difference in favour of boys has been observed in Colombia, where boys scored around 20 points higher than girls. Among the countries with a high gap, between 15 and 18 points, Italy is the only European country (Contini et al. 2017;OECD 2019a). In 43 out of 64 countries and economies, the gender gap in mathematics performance in favour of boys did not change significantly between 2009 and 2018. Notably, in Finland, Greece, Iceland, Luxembourg, the Netherlands, and Switzerland, the narrowing of the gender gap in mathematics performance observed in 2018 assessment is due to a significant decline in boys’ performance in mathematics (OECD 2019a). Looking at the different performance levels, boys are generally over-represented at both the bottom and the top of the performance distributions in mathematics (OECD 2019a). In many countries, girls’ scores in the first decile of the distribution of mathematics performance are higher than boys’ scores, meaning that the lowest-performing girls score above the lowest-performing boys in their countries. However, the largest differences are observed at the top of the distribution of mathematics performance, where an important male-oriented gender distributional imbalance among high achievers emerges (Breda et al. 2018;Zhou et al. 2017). 2.2. Conceptual Framework The factors affecting the achievement of students have been widely studied for a long time (De Witte and Kortelainen 2013). In particular, Chaman et al. (2014) presents a review of the factors affecting math performances on secondary education students. He considers in particular mathematic anxiety, attitude towards math, parental involvement, gender, and cultural differences. The present research focuses on three categories of determinants impacting students’ performance, which might have heterogeneous impacts on male and female students. The categories relate to (i) student’s family characteristics, (ii) student’s perceptions and attitudes, and (iii) class and school environment. Among student’s family characteristics, the home environment, ranging from parents’ attitude toward education to socioeconomic status, plays an important role in shaping students’ achievement of girls and boys (Bertocchi and Bozzano 2020). Steinthorsdottir and Sriraman (2008) have found that the involvement and support of families have a different effect on boys and girls: while boys benefit from a family context with high parent pressures, female students benefit more from parents showing interest in their school activities. Future plans and ambitions expressed by students are also important determinants explaining gender differences in student performance (Steinthorsdottir and Sriraman 2008). Parents’ preference for boys may also explain a gender disparity in the school support provided to their children (Dossi et al. 2021). The socioeconomic status has a greater impact on Economies 2023,11, 32 4 of 20 the PISA results in mathematics for female pupils, determining a higher gender gap for disadvantaged students (Schleicher 2019). Moreover, girls’ performance is usually better in families with working mothers, suggesting that gender identities are transmitted from mothers to daughters (González de San Román and De La Rica 2012). Confirming this result, Brenøe and Lundberg (2018) have found that girls benefit more from maternal education and employment than boys. Second, the students’ perceptions and attitudes, such as well-being and personal interests may explain a substantial part of the students’ performance in mathematics (Marsh and Martin 2011). In relation to COVID-19, studies on psychological aspects are gaining attention (Wang et al. 2022). Evidence shows that boys usually report a greater self-efficacy compared to girls (Close and Shiel 2009). Performances being equal, female students tend to underestimate their mathematical abilities than their male fellows (Sikora and Pitt 2019), and this affects their cognitive performance, motivation and attitudes, as well as future career perspectives (Aiello et al. 2021). Similarly, girls seem to be more anxious about mathematical problems and in implementing mathematical thinking (Close and Shiel 2009). Halpern and Ikier (2002) argue that this could be linked to the fact that boys have a greater experience of using math in their everyday life, compared to girls. However, females’ anxiety about math may be related to additional factors (Caviola et al. 2022) such as low levels of confidence and self-perception (Cvencek et al. 2014;Pajares 2005) or gender stereotypes regarding STEM and math achievement (Flore and Wicherts 2015; Starr and Simpkins 2021;Tomassini 2021). Finally, the positive attitude towards reading is not only an important predictor of reading performance, but it is also related to mathematics achievement, as a measure of the positive attitude of students towards learning. In this respect, (Ajello et al. 2018) demonstrate how girls are advantaged in mathematics items with a high reading demand, independent of their level of reading literacy. Third, to better understand the role of schools and teachers, it is relevant to consider variables on teacher behaviour and school characteristics (i.e., class and school environment). Previous research has shown how teachers’ beliefs and expectations about student performance differ depending on students’ gender-leading to learning gaps, usually in favour of male students and especially regarding STEM subjects (Jaremus et al. 2020;Mizala et al. 2015;Rainey et al. 2019). Rainey et al. (2019) find that active teaching environments may positively impact students’ sense of belonging and desire to continue in STEM. Bertocchi and Bozzano (2020) also point out that female students can be encouraged and engaged in studying STEM subjects by the presence of a female teacher, who may be seen as a role model and could set up curricula that are more attractive to girls. Finally, teh school environment represents a key factor in affecting gender differences in student performance, and previous studies have shown that the school peer pressure and expectations not only are very different between boys and girls, but also influence differently student behaviours and performance (Steinthorsdottir and Sriraman 2008). Moreover, Gibbs (2010) stresses the role of school curricula in enlarging the gender gap in disfavour of girls, particularly because of a content change in mathematics topics over years, which increasingly focus on topics that tend to favour boys (like spatial and logical items). 3. Data and Methods 3.1. Data and Variables’ Selection The empirical analyses are based on the PISA 2018 dataset, which provides internationally comparable data on the educational achievement of 15-year-old students, together with several background information on students, schools, and families. PISA 2018 is the last wave available, allowing exploring the most recent information on students’ achievement. As mentioned in Section 1, by analysing the math achievement of 15-year-old students, we can provide evidence of the gender gap in the last years of compulsory school. The results are, thus, particularly significant since the gender gap found at this educational stage is more likely to affect the future job career of secondary-school participants. For the same reason, we focus exclusively on students enrolled in general track schools, without consid- Economies 2023,11, 32 5 of 20 ering the ones attending a vocational track. In this way, we can provide detailed evidence on the students who are more likely to attend universities and, therefore, who would be potentially more affected by a gap in math achievement during their educational path. Based on the conceptual framework presented in Section 2.2, we study the influence of three categories of variables on the mathematics achievement of male and female students. More specifically, the three groups of variables concern the student’s family characteristics, student’s perceptions and attitudes, and class and school environment. All the indicators are based on PISA 2018 questionnaire and are described in Table 1. Table 1. Description of the student-level variables. Variable PISA Code Type Description Student’s family characteristics Mother edu ST005 and ST006 cat Indicate the highest level of education achieved by the mother and it is based on the questions ST005 and ST006. 0 = primary education not completed, 1 = complete primary education, 2 = complete lower secondary education, 3 = complete upper secondary education, 4 = complete post-secondary non tertiary education, 5 = complete tertiary education, 6 = complete postgraduate education Parent support EMOSUPS num Standardized indicator of parents’ emotional support. It was constructed by PISA on the base of question ST123 and it ranges between −2.447 and 1.035. ESCS ESCS num Standardized index of economic, social and cultural status, derived by PISA, based on the parents’ highest level of education (PARED), parents’ highest occupational status (HISEI), and home possessions (HOMEPOS), including Books in the home Foreign language ST022Q01TA 0/1 Language that the students speak at home. 0 = same language as at school, 1 = different language ICT resources ICTRES num Standardized indicator of ICT home possessions. It ranges between −3.968 and 3.612 Student’s perceptions and attitudes Fear failure GFOFAIL num Standardized indicator of the fear of failure of the student. It is based on question ST183 and it ranges between − 1.894 and 1.891 Feel awkward * ST034Q04TA 0/1 Indicator based on the sentence ‘I feel awkward and out of place in my school’. 0 = (strongly) disagree with the sentence, 1 = (strongly) agree with the sentence Feel outsider * ST034Q01TA 0/1 Indicator based on the sentence ‘I feel like an outsider (or left out of things) at school’. 0 = (strongly) disagree with the sentence, 1 = (strongly) agree with the sentence Self confidence * ST034Q05TA 0/1 Indicator based on the sentence ‘Other students seem to like me.’ 0 = (strongly) disagree with the sentence, 1 = (strongly) agree with the sentence Sociable * ST034Q02TA 0/1 Indicator based on the sentence ‘I make friends easily at school.’ 0 = (strongly) disagree with the sentence,1 = (strongly) agree with the sentence Enjoyment reading ST175Q01IA cat Time spent by the students reading for enjoyment. 0 = no time, 1 = less than 30 min per day, 2 = between 30 and 60 min per day, 3 = between 1 and 2 h, 4 = more than 2 h Class and school environment Teach support (global) TEACHSUP num Standardized indicator of teacher support, constructed by PISA on the base of question ST100. It ranges between − 2.743 and 1.341 Discipline language class DISCLIMA num Standardized indicator of disciplinary climate in the language-ofinstruction lessons, provided by PISA. It is based on ST097 and it ranges between −2.712 and 2.034. Longest book ST154Q01HA cat Number of pages of the longest text the student had to read for school. 1 = one page or less, 2 = between 2 and 10 pages, 3 = between 11 and 50 pages, 4 = between 51 and 100 pages, 5 = between 101 and 500 pages, 6 = more than 500 pages Class size CLSIZE num Number of students in the class, it ranges between 13 and 53. Perception cooperation PERCOOP num Cooperation climate perceived by students, it is a standardized indicator computed by PISA based on question ST206. and it ranges between −2.143 and 1.676 Note: Variables marked with * were originally Likert scale questions from 1 to 5, here dichotomized by the authors. Values from 1 to 3 were assigned 0, otherwise 1. Economies 2023,11, 32 6 of 20 Student’s family characteristics include: (i) the level of education of the mother (ST005 and ST006), (ii) the perceived support of the parents (EMOSUPS), (iii) an index of the socio-economic background of the student (ESCS), (iv) a binary variable indicating whether the pupil speaks a foreign language at home (ST022Q01TA), and (v) a binary variable indicating whether there are ICT resources at home (ICTRES). Regarding student’s perceptions and attitudes, we consider variables that describe students’ fear of failure (GFOFAIL), student’s feeling awkward (ST034Q05T4) or outsider (ST034Q01TA), student’s perception of being liked (or not) by other students (ST034Q05TA), and student’s ability to make friends easily (ST034Q02TA). Moreover, as described in Section 2.2, attitude towards reading is included here as it potentially explains math scores’ differences between genders (ST175Q01IA). Lastly, on the class and school environment, we consider variables concerning how much teachers are supportive towards students (TEACHSUP), how students perceive the teacher to be able to maintain discipline in the class (DISCLIMA), the number of pages of the longest book students had to read for school purposes (ST154Q01HA), the class size (CLSIZE), and the perceived climate of cooperation in the school (PERCOOP). The indicators described in Table 1are available for 28 European countries. Despite some relevant countries (such as Sweden and Norway) having been excluded from the study for problems with data availability, the analyses can provide a comprehensive overview of the European area. On the other hand, additional indicators that could potentially explain math achievement have not been considered because they report missing values for several European countries. 3.2. Methodology The aim of our analysis is to investigate the mechanisms that determine the heterogeneity in students’ performance across gender. We are interested in exploring the gender educational gap within countries and in identifying which variables are associated with females’ and males’ performance, within different contexts. To this end, our methodological approach consists of two steps. In the first step, we identify three categories of European countries: countries where males perform on average better than females (Group 1), countries where there is no evidence of a gender gap (Group 2), and countries where females perform on average better than males (Group 3). For each country, we perform a parametric two-sample t-test for comparing the means of males and females performances and, standing on the p-value, we assign the country to one of the three categories (see Table A1 in Appendix Afor details). The three categories represent three different social contexts and, in the second step, our aim is to investigate, separately for each of them, which are the most important determinants of students’ scores for boys and girls, respectively. To this end, for each category of countries c ={Group 1, Group 2, Group 3} and for each gender g = {Female, Male}, we perform a multilevel random forest (Pellagatti et al. 2021) in which we consider students (level 1), nested within countries (level 2). For each student i of gender g , attending a school in country jwithin category c, the model takes the following form: yij,gc =fgc(xij,gc) + bj,gc +eij,gc where yij,gc is the math PISA test score of student i ; xij,gc is the set of student level covariates relative to student i ; fgc(·) identifies the random forest term; bj,gc ∼ N ( 0, σ2 gc) is the random intercept relative to country j; and eij,gc ∼ N (0, ω2 gc)is the error term. We adopt this modelling for two main reasons. First, the multilevel approach allows us to take into account the countries as grouping factor and to estimate the heterogeneity in student performances net of any structural differences across countries. Educational systems could significantly influence students’ differences in performance by gender, and thus it is relevant to estimate determinants within countries. For instance, by performing an empirical analysis on 32 countries, Ayalon and Livneh (2013) show that the betweencountries variation in the gender gap in mathematics can be explained by the different levels Economies 2023,11, 32 7 of 20 of standardisation of the national educational systems. In addition, Cascella et al. (2021) show that gender differences in mathematics can be attributed to different socio-cultural and economic factors that can vary among countries and regions. Similarly, González de San Román and De La Rica (2012) and Cuevas-Ruiz et al. (2020) state that girls’ performance is better in societies where gender equality is valued. For these reasons, after the partition of the European countries within the three categories, there is still a component that varies across countries of the same category that we can quantify. Therefore, given the estimates of the variance of the random effects ˆ σ2 gc and of the error term ˆ ω2 gc , we compute the Percentage of Variability explained by the Random effects (PVRE) as ˆ σ2 gc ˆ σ2 gc+ˆ ω2 gc , that represents the percentage of the unexplained variability in student performance explained at country level. By comparing this quantity across categories of countries and across genders, we can explore the relevance of the country component by gender. In particular, for both genders, the estimate of the coefficient bj,gc quantifies the effect of country j on its female and male student performances, respectively. Second, the random forest approach allows us to estimate the effect of the covariates in a flexible and interpretable way (Masci et al. 2018;Schiltz et al. 2017). This is fundamental given the numerous predictors that we would like to consider and their potential non-linear association with the response. Parametric multilevel models require a priori knowledge to choose their parametric form and often result to be too restrictive when covariates have different types of relationships and interactions with the response. Indeed, they basically capture only relationships that have the pre-specified functional form. With respect to them, random forest allows handling a higher number of-potentially correlated-covariates and easily modelling their interactions and their different associations with the response. Random forest is an ensemble of regression trees (Breiman 2001;Friedman et al. 2001;James et al. 2013;Lewis 2000) and, given a response variable and a set of covariates, it computes an importance ranking of the covariates by measuring the ability of each covariate to improve the estimation. For each covariate, this measure, labelled as % IncMSE , is computed from permuting Out-Of-Bag (OOB) data in the following way: for each tree of the random forest, the Mean Square Error (MSE) on the OOB portion of the data is recorder; the same is then done after permuting the covariate; the difference between the two are then averaged over all trees, and normalised by the standard deviation of the differences. Besides the importance ranking of the covariates, we can further investigate the effect of each covariate on the response variable by means of Partial Dependence Plots (PDPs). For each covariate, the PDP represents the net effect of the covariate on the response, after averaging out the effect of all other covariates. 4. Results 4.1. Preliminary Results: Country Groups Based on Gender Gap’S Direction As underlined in Sections 1and 3.2, the gender gap in mathematics can differ importantly among countries. These disparities are linked to substantial heterogeneity in socioeconomic and cultural characteristics across European regions, as well as differences in educational systems. While cross-country disparities can be taken into account by the random intercept in the multilevel random forest, the work aims at exploring how the results differ across the three groups of countries that we have identified (i.e., Groups 1, 2, and 3). The analyses are, thus, performed separately for the three different groups. Table A1 in Appendix Aprovides an overview of the PISA math scores by gender for all the countries considered in the analysis and the p-values resulting from the two-sample t-test, indicating if the distributions of scores are statistically different across countries. The map in Figure 1 displays the selected countries divided into the three groups. Group 1 is the most numerous group, with 20 countries. The large number of countries in this group stresses the urgency of addressing the gender gap in disfavour of girls in most European countries. Moreover, preliminary results indicate that scores’ distributions are not statistically different across gender (Group 2) in five European countries: the Czech Republic, Switzerland, Slovakia, Economies 2023,11, 32 8 of 20 Poland, and Lithuania. Finally, Group 3 gathers the countries where females perform significantly better than males in mathematics, which are only three: Finland, Iceland, and Malta. Figure 1. Selected countries coloured standing on their assignment to the three groups based on gender gap in PISA math scores. Table 2displays the descriptive statistics of selected variables divided by gender and group and stresses different patterns. Parental support is perceived to a larger extent by countries belonging to Group 1, where females perform worse than males. In this group, girls have a higher perceived cooperation climate than males, a higher perceived discipline in the class, and a more pronounced perceived parental support. This indicates that girls tend, in general, to be more positive about the school and home environment, and this attitude possibly leads to positive spill-overs. However, on average, girls also have a higher fear of failure-revealing that positive attitudes regarding the environment are not translated into better self-esteem. Other characteristics for which male and female populations differ are the ones related to ICT and reading. In particular, boys report having greater access to ICT at home, while girls read more and, mostly, enjoy more reading. 4.2. Main Results: Multilevel Random Forest In this section, we describe the main findings emerging from the multilevel random forest models. In particular, six models, obtained by grouping countries based on their gender gap and by gender, are computed. For each model, the country effect and studentlevel variables’ importance are shown in Table 3and reported in detail in the Appendix A (see Figures A2 and A1). Table 3shows the ranking’s position of the covariates and the related Inc%MSE for each model (by gender and by context’s country group). To facilitate the reading, results about student and country levels are presented separately in the following sections. Economies 2023,11, 32 15 of 20 Author Contributions: Conceptualization, A.B., M.C., M.D.L., C.M., A.M., L.R., and M.S.; methodology, M.D.L., C.M., A.M., and L.R.; formal analysis, M.D.L., C.M., A.M., and L.R.; investigation, A.B., M.C., and M.S.; data curation, M.D.L., C.M., A.M., and L.R.; writing original draft preparation, A.B., M.C., and M.S.; writing review and editing, A.B., M.C., M.D.L., C.M., A.M., L.R., and M.S. All authors have read and agreed to the published version of the manuscript. Funding: Anna Mergoni is grateful to FWO for the financial support (grant number 11G5520N). Institutional Review Board Statement: Ethical review and approval were waived for this study, given that the data analysed are open sourced and properly anonymised. Data Availability Statement: The data used for this paper are available at the OECD following link: https://www.oecd.org/pisa/data/, accessed on 15 December 2021. Conflicts of Interest: The authors declare no conflict of interest. Appendix A Table A1. Details of gender differences among countries by PISA math scores. Country p.Value Mean Score Mean Score # obs. Group for Females for Males AUT 0 496.979 507.410 6802 better male BEL 0 541.043 560.719 4888 better male BGR 0.004 464.893 475.523 2739 better male HRV 0 508.091 541.658 2094 better male CZE 0.546 533.560 531.863 4764 no gap DNK 0.00004 493.128 501.081 7656 better male EST 0.005 519.645 525.908 5315 better male FIN 0.005 510.455 504.353 5648 better female FRA 0.00002 499.445 510.525 4992 better male DEU 0.002 499.226 507.447 5305 better male GRC 0.0003 462.327 470.351 5599 better male HUN 0 494.471 511.977 4294 better male ISL 0.008 498.026 489.664 3296 better female IRL 0.005 498.221 504.029 5536 better male ITA 0 508.668 545.193 5744 better male LVA 0.001 490.650 498.148 5259 better male LTU 0.480 481.024 482.566 6758 no gap LUX 0.0004 489.381 500.135 4123 better male MLT 0.0003 480.116 467.605 3363 better female NLD 0.0001 547.000 557.371 3379 better male POL 0.204 515.407 518.377 5616 no gap PRT 0 496.962 510.137 4965 better male ROU 0 435.553 449.897 4437 better male SVK 0.775 495.028 494.051 3900 no gap SVN 0.00005 545.966 560.069 2221 better male ESP 0 488.831 495.895 35,599 better male CHE 0.389 513.383 515.740 4841 no gap GBR 0 491.066 498.559 13,762 better male Economies 2023,11, 32 16 of 20 (a) (b) (c) (d) (e) (f) Figure A1. Variable importance plots. ( a ) Females—Countries where males perform better than females. ( b ) Males—Countries where males perform better than females. ( c ) Females—Countries with no gap. ( d ) Males—Countries with no gap. ( e ) Females—Countries where females perform better than males. (f) Males—Countries where females perform better than males. (a) (b) Figure A2. Cont. Economies 2023,11, 32 17 of 20 (c) (d) (e) (f) Figure A2. Dotplot of country effect. ( a ) Females—Countries where males perform better than females.( b ) Males—Countries where males perform better than females. ( c ) Females—Countries with no gap. ( d ) Males—Countries with no gap. ( e ) Females—Countries where females perform better than males. (f) Males—Countries where females perform better than males. Note 1 We report in the paper the partial plots only for Group 1 since this includes the majority of the countries and represents the group of interest in terms of policy action, i.e., countries with a gender gap in disfavour of girls. Partial plots for Groups 2 and 3 show similar patterns and are available upon request References Ajello, Anna Maria, Elisa Caponera, and Laura Palmerio. 2018. Italian students’ results in the pisa mathematics test: Does reading competence matter? European Journal of Psychology of Education 33: 505–20. [CrossRef] Ajello, Anna Maria, Giulia Accardi, Stefano Aprile, Rosalia Caldarella, Ciriaco Carru, Marcello Ciaccio, Immaculata De Vivo, Caterina Maria Gambino, Mattia Emanuela Ligotti, and Sonya Vasto. 2021. Age and gender-related variations of molecular and phenotypic parameters in a cohort of sicilian population: From young to centenarians. Aging and Disease 12: 1773. Autor, David, David Figlio, Krzysztof Karbownik, Jeffrey Roth and Melanie Wasserman. 2019. Family disadvantage and the gender gap in behavioral and educational outcomes. American Economic Journal: Applied Economics 11: 338–81. [CrossRef] Ayalon, Hanna, and Idit Livneh. 2013. Educational standardization and gender differences in mathematics achievement: A comparative study. Social Science Research 42: 432–45. [CrossRef] Beede, David N., Tiffany A. Julian, David Langdon, George McKittrick, Beethika Khan, and Mark E. Doms. 2011 Women in STEM: A Gender Gap to Innovation. SSRN Electronic Journal. [CrossRef] Bertocchi, Graziella, and Monica Bozzano. 2020. Gender Gaps in Education. Cham: Springer International Publishing. Borgonovi, Francesca, Álvaro Choi, and Marco Paccagnella. 2021. The evolution of gender gaps in numeracy and literacy between childhood and adulthood. Economics of Education Review [CrossRef] Bouffard, Suzanne M., and Nancy E. Hill. 2005. Maternal perceptions of competence and children’s academic adjustment: Longitudinal relations across early elementary school. Social Psychology of Education 8: 441–63. [CrossRef] Breda, Thomas, and Clotilde Napp. 2019. Girls’ comparative advantage in reading can largely explain the gender gap in math-related fields. Proceedings of the National Academy of Sciences 116: 15435–40. [CrossRef] Economies 2023,11, 32 18 of 20 Breda, Thomas, Elyès Jouini, and Clotilde Napp. 2018. Societal inequalities amplify gender gaps in math. Science 359: 1219–20. [CrossRef] Breiman, Leo. 2001. Random forests. Machine Learning 45: 5–32. [CrossRef] Brenøe, Anne Ardila, and Shelly Lundberg. 2018. Gender gaps in the effects of childhood family environment: Do they persist into adulthood? European Economic Review 109: 42–62. [CrossRef] Broer, Markus, Yifan Bai, and Frank Fonseca. 2019. A review of the literature on socioeconomic status and educational achievement. In Socioeconomic Inequality and Educational Outcomes. Cham: Springer International Publishing, vol. 5, pp. 7–17. [CrossRef] Buchmann, Claudia, Thomas A. DiPrete, and Anne McDaniel. 2008. Gender inequalities in education. Annu. Rev. Sociol 34: 319–37. [CrossRef] Card, David, and A. Abigail Payne. 2021. High school choices and the gender gap in stem. Economic Inquiry 59: 9–28. [CrossRef] Cascella, Clelia, Julian Scott Williams, and Maria Pampaka. 2021. Gender differences in mathematics outcomes at different levels of locality to inform policy and practice. European Educational Research Journal 21: 705–31 . [CrossRef] Caviola, Sara, Enrico Toffalini, David Giofrè, Jessica Mercader Ruiz, Dénes Sz˝ucs, and Irene C. Mammarella. 2022. Math performance and academic anxiety forms, from sociodemographic to cognitive aspects: A meta-analysis on 906,311 participants. Educational Psychology Review 34: 363–99. [CrossRef] Chaman, Mini J., Kim Beswick, and Rosemary Callingham. 2014. Factors influencing mathematics achievement among secondary school students: A review. In The Future of Educational Research. Rotterdam: SensePublishers, pp. 227–38. Close, Sean, and Gerry Shiel. 2009. Gender and pisa mathematics: Irish results in context. European Educational Research Journal 8: 20–33. [CrossRef] Contini, Dalit, Maria Laura Di Tommaso, and Silvia Mendolia. 2017. The gender gap in mathematics achievement: Evidence from italian data. Economics of Education Review 58: 32–42. [CrossRef] Cuevas-Ruiz, Pilar, Cristina Borra, and Almudena Sevilla. 2020. Educación y salud al nacer. Papeles de Economía Española 166: 185–202. Available online: https://www.funcas.es/wp-content/uploads/2021/01/Pilar-Cuevas-Cristina-Borra-Almudena-Sevilla.pdf (accessed on 21 June 2022) Cvencek, Dario, Andrew N. Meltzoff, and Manu Kapur. 2014. Cognitive consistency and math–Gender stereotypes in singaporean children. Journal of Experimental Child Psychology 117: 73–91. [CrossRef] De Witte, Kristof, and Mika Kortelainen. 2013. What explains the performance of students in a heterogeneous environment? conditional efficiency estimation with continuous and discrete environmental variables. Applied Economics 45: 2401–12. [CrossRef] Dossi, Gaia, David Figlio, Paola Giuliano, and Paola Sapienza. 2021. Born in the family: Preferences for boys and the gender gap in math. Journal of Economic Behavior & Organization 183: 175–88. Dunnzlaff, Lina, Dirk Neumann, Judith Niehues, and Andreas Peichl. 2011. Equality of Opportunity and Redistribution in Europe. Bingley: Emerald Group Publishing Limited. Education, European, Culture Executive Agency, and Eurydice. 2012. Gender Differences in Educational Outcomes: Study on the Measures Taken and the Current Situation in Europe. Publications Office. Available online: https://data.europa.eu/doi/10.2797/3598 (accessed on 21 June 2022) . Else-Quest, Nicole M., Janet Shibley Hyde, and Marcia C. Linn. 2010. Cross-national patterns of gender differences in mathematics: A meta-analysis. Psychological Bulletin 136: 103. [CrossRef] Evans, David K., Maryam Akmal, and Pamela Jakiela. 2020. Gender Gaps in Education: The Long View. Technical Report. Washington, DC: Center for Global Development. Flore, Paulette C., and Jelte M Wicherts. 2015. Does stereotype threat influence performance of girls in stereotyped domains? A meta-analysis. Journal of School Psychology 53: 25–44. [CrossRef] [PubMed] Friedman, Jerome, Trevor Hastie, and Robert Tibshirani. 2001. The Elements of Statistical Learning. Springer Series in Statistics. New York: Springer, vol. 1. Fryer, Roland G., Jr., and Steven D. Levitt. 2010. An empirical analysis of the gender gap in mathematics. American Economic Journal: Applied Economics 2: 210–40. [CrossRef] Gevrek, Z. Eylem, Deniz Gevrek, and Christian Neumeier. 2020. Explaining the gender gaps in mathematics achievement and attitudes: The role of societal gender equality. Economics of Education Review 76: 101978. [CrossRef] Ghaith, Ghazi M. 2002. The relationship between cooperative learning, perception of social support, and academic achievement. System 30: 263–73. [CrossRef] Gibbs, Benjamin G. 2010. Reversing fortunes or content change? gender gaps in math-related skill throughout childhood. Social Science Research 39: 540–69. [CrossRef] Giuliano, Paola. 2020. Gender and culture. Oxford Review of Economic Policy 36: 944–61. [CrossRef] González de San Román, Ainara, and Sara De La Rica. 2012. Gender Gaps in Pisa Test Scores: The Impact of Social Norms and the Mother’s Transmission of Role Attitudes. IZA DP No. 6338. Available online: https://docs.iza.org/dp6338.pdf (accessed on 21 June 2022) Halpern, Diane F., and Simay Ikier. 2002. Biology, Society, and Behavior: The Development of Sex Differences in Cognition. Westport: Ablex Publishing, pp. 3–19. James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning. Berlin/Heidelberg: Springer, vol. 112. Economies 2023,11, 32 19 of 20 Jaremus, Felicia, Jennifer Gore, Elena Prieto-Rodriguez, and Leanne Fray. 2020. Girls are still being ‘counted out’: Teacher expectations of high-level mathematics students. Educational Studies in Mathematics 105: 219–36. [CrossRef] Jerrim, John, Luis Alejandro Lopez-Agudo, and Oscar D. Marcenaro-Gutierrez. 2020. Does it matter what children read? new evidence using longitudinal census data from spain. Oxford Review of Education 46: 515–33. [CrossRef] Lee, Jaekyung. 2012. Educational equity and adequacy for disadvantaged minority students: School and teacher resource gaps toward national mathematics proficiency standard. The Journal of Educational Research 105: 64–75. [CrossRef] Lee, Valerie E., Julia B. Smith, and Robert G. Croninger. 1997. How high school organization influences the equitable distribution of learning in mathematics and science. Sociology of Education 70: 128–50. [CrossRef] Levine, Susan C., Marina Vasilyeva, Stella F. Lourenco, Nora S. Newcombe, and Janellen Huttenlocher. 2005. Socioeconomic status modifies the sex difference in spatial skill. Psychological Science 16: 841–45. [CrossRef] [PubMed] Lewis, Roger J. 2000. An introduction to classification and regression tree (cart) analysis. Paper presented at Annual Meeting of the Society for Academic Emergency Medicine, San Francisco, CA, USA. May 22–25. Machin, Stephen, and Patrick A Puhani. 2003. Subject of degree and the gender wage differential: Evidence from the uk and germany. Economics Letters 79: 393–400. [CrossRef] Marsh, Herbert W., and Andrew J. Martin. 2011. Academic self-concept and academic achievement: Relations and causal ordering. British Journal of Educational Psychology 81: 59–77. [CrossRef] Martins, Lurdes, and Paula Veiga. 2010. Do inequalities in parents’ education play an important role in pisa students’ mathematics achievement test score disparities? Economics of Education Review 29: 1016–33. [CrossRef] Masci, Chiara, Geraint Johnes, and Tommaso Agasisti. 2018. Student and school performance across countries: A machine learning approach. European Journal of Operational Research 269: 1072–85. [CrossRef] Mejias, Paulina Perez, Dora Elias McAllister, Karina G. Diaz, and Javiera Ravest. 2021. A longitudinal study of the gender gap in mathematics achievement: evidence from chile. Educational Studies in Mathematics 107: 583–605. [CrossRef] Mizala, Alejandra, Francisco Martínez, and Salomé Martínez. 2015. Pre-service elementary school teachers’ expectations about student performance: How their beliefs are affected by their mathematics anxiety and student’s gender. Teaching and Teacher Education 50: 70–78. [CrossRef] Moss, Gemma, and John W. McDonald. 2004. The borrowers: Library records as unobtrusive measures of children’s reading preferences. Journal of Research in Reading 27: 401–12. [CrossRef] Munir, Farzana, and Rudolf Winter-Ebmer. 2018. Decomposing international gender test score differences. Journal for Labour Market Research 52: 1–17. [CrossRef] OECD. 2016. PISA 2015 Results (Volume I). Paris: OECD. [CrossRef] OECD. 2019a. PISA 2018 Results (Volume II). Paris: OECD. [CrossRef] OECD. 2019b. PISA 2018 Results (Volume III): What School Life Means for Students’ Lives. Paris: OECD. Pajares, Frank. 2005. Gender Differences in Mathematics Self-Efficacy Beliefs. Cambridge: Cambridge University Press. Pellagatti, Massimo, Chiara Masci, Francesca Ieva, and Anna M. Paganoni. 2021. Generalized mixed-effects random forest: A flexible approach to predict university student dropout. Statistical Analysis and Data Mining: The ASA Data Science Journal 14: 241–57. [CrossRef] Piazzalunga, Daniela. 2018. The gender wage gap among college graduates in italy. Italian Economic Journal 4: 33–90. [CrossRef] Rainey, Katherine, Melissa Dancy, Roslyn Mickelson, Elizabeth Stearns, and Stephanie Moller. 2019. A descriptive study of race and gender differences in how instructional style and perceived professor care influence decisions to major in stem. International Journal of STEM Education 6: 1–13. [CrossRef] Schiltz, Fritz, Chiara Masci, Tommaso Agasisti, and Dániel Horn. 2017. Using Machine Learning to Model Interaction Effects in Education: A Graphical Approach. MOX-Report No. 31/2017. Available online: https://www.mate.polimi.it/biblioteca/add/ qmox/31-2017.pdf (accessed on 21 June 2022). Schleicher, Andreas. 2019. Pisa 2018: Insights and Interpretations. Paris: OECD Publishing. Sikora, Joanna, and David GW Pitt. 2019. Does advanced mathematics help students enter university more than basic mathematics? gender and returns to year 12 mathematics in australia. Mathematics Education Research Journal 31: 197–218. [CrossRef] Starr, Christine R., and Sandra D. Simpkins. 2021. High school students’ math and science gender stereotypes: Relations with their stem outcomes and socializers’ stereotypes. Social Psychology of Education 24: 273–98. [CrossRef] Steinthorsdottir, Olof Bjorg, and Bharath Sriraman. 2008. Exploring gender factors related to pisa 2003 results in iceland: A youth interview study. ZDM 40: 591–600. [CrossRef] Tomassini, Cecilia. 2021. Gender gaps in science: Systematic review of the main explanations and research agenda. Education in the Knowledge Society (EKS) 22: 25. [CrossRef] Torche, Florencia. 2015. Analyses of intergenerational mobility: An interdisciplinary review. The ANNALS of the American Academy of Political and Social Science 657: 37–62. [CrossRef] UNICEF. 2020. Towards an Equal Future: Reimagining Girls’ Education through Stem. Available online: https://www.unicef.org/ media/84046/file/Reimagining-girls-education-through-stem-2020.pdf (accessed on 25 June 2022) van Hek, Margriet, Claudia Buchmann, and Gerbert Kraaykamp. 2019. Educational systems and gender differences in reading: A comparative multilevel analysis. European Sociological Review 35: 169–86. [CrossRef] Economies 2023,11, 32 20 of 20 Wang, Yurou, Mengya Xia, Wenjing Guo, Fangjie Xu, and Yadan Zhao. 2022. Academic performance under covid-19: The role of online learning readiness and emotional competence. Current Psychology. [CrossRef] Zhou, Yisu, Xitao Fan, Xiaoxin Wei, and Robert H. Tai. 2017. Gender gap among high achievers in math and implications for stem pipeline. The Asia-Pacific Education Researcher 26: 259–69. [CrossRef] Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.