On the size of sheepskin effects: A meta-analysis

Mora Rodríguez, Jhon James; Muro, Juan Working Paper On the size of sheepskin effects: A meta-analysis Economics Discussion Papers, No. 2015-31 Provided in Cooperation with: Kiel Institute for the World Economy – Leibniz Center for Research on Global Economic Challenges Suggested Citation: Mora Rodríguez, Jhon James; Muro, Juan (2015) : On the size of sheepskin effects: A meta-analysis, Economics Discussion Papers, No. 2015-31, Kiel Institute for the World Economy (IfW), Kiel This Version is available at: https://hdl.handle.net/10419/109942 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/3.0/ Received January 15, 2015 Accepted as Economics Discussion Paper March 6, 2015 Published April 23, 2015 © Author(s) 2015. Licensed under the Creative Commons License - Attribution 3.0 Discussion Paper No. 2015-31 | April 23, 2015 | http://www.economics-ejournal.org/economics/discussionpapers/2015-31 On the Size of Sheepskin Effects: A Meta-Analysis Jhon James Mora Rodríguez and Juan Muro Abstract The authors use information gathered from 122 studies on the effects of high school diplomas on wages in different countries worldwide to carry out a meta-analysis that shows high school diplomas have a statistically significant effect on wages of nearly 8 percent. This effect varies whether the country is away from the tropics or whether factors such as sex, race, and continent are taken into account. The authors’ results also reveal the existence of a publication bias that tends to increase the magnitude of the sheepskin effect. Nevertheless, when the former is factored into the analysis the latter remains statistically significant. (Published in Special Issue Meta-Analysis in Theory and Practice) JEL C8 I21 J24 Keywords Sheepskin effects; meta-analysis; publication bias Authors Jhon James Mora Rodríguez, Universidad Icesi, Colombia, [email protected] Juan Muro, Universidad de Alcalá, Spain Citation Jhon James Mora Rodríguez and Juan Muro (2015). On the Size of Sheepskin Effects: A Meta-Analysis. Economics Discussion Papers, No 2015-31, Kiel Institute for the World Economy. http://www.economics-ejournal.org/ economics/discussionpapers/2015-31 2 1. Introduction The degree equation was first developed by Hungerford and Solon in 1987 and is usually known as the “sheepskin effect equation”. The equation, which is estimated from a regression of individuals’ wages in a given country, is aimed at determining the effects of school diplomas and degrees on wages in a specific setting. Using cross-sectional data, Hungerford and Solon (1987) found that there is a return for each year of education and an additional significant return on the years during which a diploma or degree is earned. Since then many studies have been carried out to test the hypothesis and measure the sheepskin effect. For our review most of this research was completed in Brazil (29.51%), the United States (24.59%), and Colombia (10.66%). While it is evident that there may be measurement errors in educational attainment when empirical research is based on data of self-reported education levels (see Card, 1999; Kane et al., 1999) and that ordinary least squares (OLS) estimates overstate the effects of a diploma/degree, we also must observe that if sheepskin effects persist across different countries, their importance should not be neglected. The existence of diploma/degree effects is obviously important when it comes to establishing educational policies in any country because of the high social costs involved, particularly in developing countries. One of the possible ways to determine the magnitude of sheepskin effects is by examining various publications and working papers on this subject. In this paper we conduct a meta-analysis of the diploma/degree equation that centers specifically on the effect of high school diplomas. We have reviewed a total of 122 published articles and working papers that cover 15 different countries, including, among others, Libya, the Philippines, and Egypt. Our findings show that the effect of a schooling degree is not only statistically significant but depends on factors such as closeness to the tropics, sex, race, and continent. The paper provides an important contribution in that it shows that the effect of a high school diploma on wages is real in a statistical sense. In other words, the said effect is not statistically equal to zero. Additionally, we find that the size of the sheepskin effect is around 8% in the case of high school diplomas. The paper is organized as follows: section 2 presents the diploma/degree equation and the metaanalysis technique; section 3 discusses relevant data; section 4 reviews the results; and the last, section provides the conclusions. 2. Sheepskin effects and meta-analysis In general, additional earnings from the complete range from school diplomas and certificates to Ph.D. degrees can be estimated from the following wage regression:          2 1 2 3 , , , 11 ln( ) exp exp ( ) TT i i i i t i t i t t i i tt Wh S DS S S S       , (1) where ln(Wh) is the logarithm of hourly wages; S is the number of years of schooling; exp and exp2 represent an individual's years of labor experience and its square; DSt is a dummy variable for the year in which a given degree is earned; St is the year in which a degree is earned; and t is the 3 credential itself, which can be as advanced as a doctoral degree (PhD). In this article we only consider the high school diploma, and not the number of years to obtain the diploma. For example in Colombia the diploma is obtained after 11 years of schooling while in the USA it takes 12 years and in the UK 13 years of the schooling. The regression in (1) allows us to estimate a β value for each schooling diploma and its standard error (Hungerford and Solon, 1987; Mora and Muro, 2008). Meta-analysis has been used in medical and psychological studies on a regular basis (see, e.g., Sterling, 1959; Rosenthal, 1979; Begg and Berlin. 1988; Borenstein et al., 2009). It has also been utilized in economics by a number of authors, including among others Card and Krueger (1995a, 1995b) to study the effects of minimum wages; Dalhuisen et al. (2003) to analyze income elasticity of water demand; Jarrell and Stanley (2004) to review wage discrimination; Abreu et al. (2005) to quantify beta-type convergence; and Colegrave and Giles (2008) to study school cost functions. Let us assume there is an article or working paper that provides information about the size of the effect of a high school diploma. Each publication also supplies information about the estimated standard error. Thus, 22 22 22 HS-Sheepskin ( , ); ( , ) H ~~ ~S-Sheepskin ( , ) HS-Sheepskin ; (0, ); (0, )~~ i i i i i i i i i i i i i i i NN N u u N N                   . (2) In equation (2), HS-sheepskin is the estimated effect derived from equation (1). It is worth noting that  i is the effect of a high school diploma, which varies from one study to another. It is assumed that it has a normal distribution around the mean effect  . The between-studies variance,  2, is estimated from relevant data and is determined using the method of moments (DerSimonian and Laird, 1986), from the following equation: (3) where Wi is the weight of each article or working paper and n is the number of articles and working papers. Equation (2) shows that the effect of a diploma could be explained with both a fixed-effect model and a random-effect model. However it does not provide any explanation as to the determinants of 2 21 1 1 2 2 ( 1) n ii ni ii n ii i i i i WSheepskin WSheepskin n W W WW                        4 variability between studies. To take into account factors that determine the variability between studies a vector of covariates Xi is incorporated, as shown in equation (4) below. (4) In (4)  i is again the estimated effect, which varies from one study to another. It is assumed that it has a normal distribution around the linear predictor of  .  2, on the other hand, is the betweenstudies variance which is estimated from relevant data and cannot be accounted for by covariates. Estimating (3) or (4) provides an initial estimate of  . Available literature on the topic of meta-analysis provides discussions of whether the aforementioned value could be biased due to the current publication policies of scientific journals. As an illustrative example, Card and Krueger (1995) and Stanley (2005) contend that there are at least three different sources of publication bias in economics: “1 – Reviewers and editors may be predisposed to accept papers consistent with the conventional view. 2 – Researchers may use the presence of a conventionally expected result as a model selection test. 3 – Everyone may possess a predisposition to treat `statistically significant` results more favorably” Stanley (2005, 310–11) To tackle this problem a test to identify the potential existence of the aforementioned publication bias has been proposed. The test is based on running the following regression: 10 effecti i i Sd e    (5) where effecti is the effect of a school diploma on wages and Sdi is its standard error. In the absence of publication bias, the estimate of the true effect will have a value close to β 1, regardless of the standard error. The distribution in equation (5), however, is heteroscedastic. A heteroscedasticitycorrected regression is obtained by transforming the standard error: 01 (1/ ) i i i t Sd e     (6) Egger et al. (1997) posit that a test of significance of β 0 is a test of publication bias that indicates the direction of the bias. Stanley (2008), on the other hand, argues that the observed effect comes close to θ when the number of observations tends to infinity and Sd tends to zero. Therefore, a test of β 1 is a test for a true effect of a school diploma that goes beyond the systematic "contamination" that arises from publication biases. Hence, β 1 is the “true” value of the effect of a school diploma. 3. Data 22 22 ( , ) ; ( ´ , ) ´ ; (0, ); (0, ) i i i i i i i i i i i i i HS Sheepskin N N X HS Sheepskin X u u N N                 5 A search on JSTOR, SCOPUS, ISI-Web, EBSCO, and GOOGLE yielded a list of 122 articles and/or papers published between 1987 and 2011. Table 1 contains summary statistics of our sample. Table 1. Data of Studies Variable Percentage (%) n Sex 44 122 Race 32 122 The Americas 72 122 By Country Beta (High School) Standard Deviation (High School) n Brazil 0.34 0.08 36 Canada 0.05 0.005 7 Colombia 0.12 0.02 13 Egypt 0.16 0.15 2 Spain 0.34 0.10 10 United States 0.09 0.05 30 The Philippines 0.13 0.03 2 Japan 0.20 0.06 2 Libya 0.16 0.08 1 Mexico 0.10 0.02 2 New Zealand 0.07 0.08 6 Pakistan 0.28 0.41 3 The Czech Republic 0.22 0.08 4 Czechoslovakia 0.19 0.10 2 Sweden 0.05 0.01 2 Weighted Average or Total 0.20 0.07 122 Source: Authors’ computation. On average, publications on the topic of sheepskin effects of a high school diploma show an additional return on a schooling degree of 19.8% with a standard deviation of 0.07. Brazil, where most studies have been carried out, is the country that evidences the greatest additional return on a school diploma. Canada and Sweden, on the other hand, are the countries with the lowest additional return. 44% of the studies consider gender differences (male vs. female), while 31% of the studies incorporate race differences (white vs. black, mestizo, and/or indigenous populations). Lastly, 72% of all studies were performed in countries on the American continents. When we compute effect/Sd the results show a minimum value of 0.01 and a maximum value of 33.75. At the 5% level of significance, 24 studies (19%) reject the sheepskin effect hypothesis, while 32 studies (26%) reject the hypothesis of the sheepskin effect at the 1% level of significance. 4. Results We carry out a meta-analysis in order to examine whether the studies share a common estimate for the effect of high school diplomas, in which case the fixed-effect method should be used, or whether there is a remarkable study heterogeneity, in which case the random-effect method should be employed. Table 2. Random and Fixed Meta-Analysis 6  2 95% Confidence Interval Z (value) I2 Number of Studies Method Β Lower Upper Fixed 0.08 0.08 0.09 54.9 90.7% 122 Random 0.15 0.003 0.13 0.16 22.3 90.7% 122 Source: Authors’ computation. Table 2 shows an estimated value of the school diploma effect of 7.9% when the fixed-effect method is used, while the estimated value is 14.5% with the random-effect model, and the between-studies variance is close to 0.03. Although both estimates of the effect of a high school diploma are statistically significant, various studies in different places around the world and the estimates for men and women or people of different races show that there is a large heterogeneity from one study to another. Therefore, the random-effect method should be used for the analysis. In order to explore the issue of heterogeneity, a Q test of heterogeneity (Borenstein et al. 2009) was carried out yielding a value of 1307.384. Under the null hypothesis that the studies share an effect in common the test follows a chi-squared distribution with k − 1 degrees of freedom. The rejection of the null hypothesis reinforces the appropriateness of using the random-effect method. Higgins et al. (2003) use an index that aims to identify to what extent the variance is spurious and to what extent it is real. Their index, I2, is on a relative scale ranging from 0 to 100 that is independent of the number of studies. If I2 is close to zero, the observed variance is largely spurious, but if I2 is close to 100, it makes sense to draw conjectures about the variance and about factors that could explain it. In other words, it is reasonable to carry out meta-regressions or subset-based analyses. Hence, according to our results in Table 2, it would make sense to incorporate covariates into our analysis. The set of covariates included in our model are the distance to the equator, a dummy variable for men (sex), a dummy variable for race (race), and a dummy variable for the Americas. The estimation results are listed in Table 3 below. Table 3. Meta-Regressions Meta-Reg[1] Meta-Reg[2] Meta-Reg[3] Meta-Reg[4] Diff-Latitude −0.289*** −0.217*** −0.141** −0.174*** (0.065) (0.046) (0.042) (0.042) Sex 0.149*** 0.079*** 0.066** (0.017) (0.020) (0.020) Race 0.114*** 0.132*** (0.025) (0.025) The Americas −0.060** (0.020) Constant 0.259*** 0.170*** 0.139*** 0.199*** (0.024) (0.018) (0.016) (0.025) 7 τ2 0.009 0.004 0.002 0.002 Q 953.6 780.9 640.2 634.7 I2 0.874 0.848 0.816 0.816 R2, adjusted 0.230 0.672 0.823 0.825 n 122 122 122 122 Source: Authors’ computation. Note: * p < 0.05, ** p < 0.01, *** p < 0.001. Table 3 shows that the effect of a high school diploma decreases as the distance to the equator increases, is larger for men than for women and, when the race variable is included in the model, is greater for white people than for black, indigenous, and other populations. With respect to the geographic variable, the studies conducted on the American continents reveal that a diploma is recognized to a lesser extent than in other countries. 4.1. Publication biases and the true effect of a high school diploma So far we have obtained estimates that as mentioned above are likely affected by publication bias. In order to test this hypothesis, we estimate the parameters in equations (5) and (6) above. Our results are in Table 4. Table 4. Publication Bias Estimates Publ. Bias[1] Publ. Bias[2] Meta-Significance Sheepskin Effect 0.997* 0.058*** (0.392) (0.014) ln(n) 0.487*** (0.055) Constant 0.130*** 2.228*** −3.330*** (0.023) (0.382) (0.552) Log-Likelihood 67.21 −295.82 −147.68 Adj. R2 0.195 0.538 0.399 Number of Cases 122 122 122 Source: Authors’ computation. Note: * p < 0.05, ** p < 0.01, *** p < 0.001. Equation (5) estimates are shown in the first column of Table 4. They suggest an effect of a high school diploma around 100%, which would mean a high school diploma would increase wages by 100%. The bias direction is positive (constant), which would imply most studies tend to report a larger 8 effect than actually observed. Due to the presence of heteroscedasticity in equation (5) we use in our analysis equation (6) estimates in the second column of Table 5. They show a much more moderate effect of a high school diploma on wages of 5.8%. In order to discuss whether the effect as so far estimated is true, we run a regression between the tvalues of each study and the sample size of the study (n). As shown by Stanley (2005), if there is indeed a true effect of high school diplomas, and given that t = β /Sd when β ≠ 0, in the regression ln(t) =  0 +  1 ln(n) the value of  1 will be statistically equal to ½. Our estimated value was 0.487 (third column in Table 4), and F for the hypothesis  1 = ½ was 0.06. This means that the observed effect of diplomas is statistically far from zero, which shows that the effect is true. To solve the problem of measurement error in equation (5) (see Sterne et al., 2000; Macaskill et al., 2001) we estimate equation (6) with IV regression using as instrumental variable the inverse of the square root of the number of observations (Stanley, 2005). Table 5. Publication Bias–Corrected Estimates Bias-corrected (1) Bias-corrected (2) Sheepskin Effect 0.088*** (0.017) Sheepskin Effect 0.079*** (0.012) (ISI or Scopus)/Sd −0.051*** (0.011) (Year of publication − 1987)/Sd −0.002* (0.001) Direction -Bias 1.126* 1.300*** (0.449) (0.337) Adj. R2 0.397 0.551 Number of Cases 122 122 Source: Authors’ computation. Note: * p < 0.05, ** p < 0.01, *** p < 0.001. The results in Table 5 show again that the bias is positive and that the “true” effect is close to 9%. 1 In column (2) we incorporate a dummy variable for ISI or Scopus journal to take into account differences arising from the quality of the publication. The result shows that if the paper was published in an ISI or Scopus journal the estimated sheepskin effect diminishes to only 2% (0.07 − 0.05). Finally, we incorporate a variable to capture the likely obsolescence of the “sheepskin effect” paradigm and its impact on the size of estimated effect. To do that we construct a time-to-origin variable calculated as the time gap between the publication year of each study and the publication year of the seminal paper by Hungerford and Solon (i.e., year of publication − 1987). In this case our results show a reduction of 0.2% per year of the sheepskin effects. 2 1 A regression was also carried out with the 30th, 60th, and 90th percentiles of the distribution. The IV-quantile regression does not yield statistically different results between percentiles [F for the difference between percentiles 30 and 60 was 1.5 with a probability of 0.223, F for the difference between percentiles 30 and 90 was 0.04 with a probability of 0.83, and F for the difference between percentiles 60 and 90 was 0 with a probability of 0.979]. 2 The total effect over the 24 years since the first publication of the sheepskin equation is around −4.8% (= −0.2% × 24).