Long-term impacts of conditional cash transfers in Latin America: Review of the evidence

Molina Millan, Teresa; Barham, Tania Catherine Jane; Macours, Karen; Maluccio, John A.; Stampini, Marco Working Paper Long-term impacts of conditional cash transfers in Latin America: Review of the evidence IDB Working Paper Series, No. IDB-WP-732 Provided in Cooperation with: Inter-American Development Bank (IDB), Washington, DC Suggested Citation: Molina Millan, Teresa; Barham, Tania Catherine Jane; Macours, Karen; Maluccio, John A.; Stampini, Marco (2016) : Long-term impacts of conditional cash transfers in Latin America: Review of the evidence, IDB Working Paper Series, No. IDB-WP-732, Inter-American Development Bank (IDB), Washington, DC, https://doi.org/10.18235/0000470 This Version is available at: https://hdl.handle.net/10419/173822 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by-nc-nd/3.0/igo/legalcode Long-Term Impacts of Conditional Cash Transfers in Latin America: Review of the Evidence Teresa Molina-Millan Tania Barham Karen Macours John A. Maluccio Marco Stampini IDB WORKING PAPER SERIES Nº IDB-WP-732 October 2016 Social Protection and Health Division Inter-American Development Bank October 2016 Long-Term Impacts of Conditional Cash Transfers in Latin America: Review of the Evidence Teresa Molina-Millan Tania Barham Karen Macours John A. Maluccio Marco Stampini Cataloging-in-Publication data provided by the Inter-American Development Bank Felipe Herrera Library Long-term impacts of conditional cash transfers in Latin America: review of the evidence / Teresa Molina-Millan, Tania Barham, Karen Macours, John A. Maluccio, Marco Stampini. p. cm. — (IDB Working Paper Series ; 732) Includes bibliographic references. 1. Transfer payments-Latin America. 2. Income maintenance programs-Latin America. 3. Economic assistance, Domestic-Latin America. 4. Poverty-Government policy-Latin A merica. I. Molina-Millan, Teresa. II. Barham, Tania. III. Macours, Karen. IV. Maluccio, John A. V. Stampini, Marco. VI. Inter-American Development Bank. Social Protection and Health Division. VII. Series. IDB-WP-732 http://www.iadb.org/SocialProtection [email protected] Copyright © Inter-American Development Bank. This work is licensed under a Creative Commons IGO 3.0 AttributionNonCommercial-NoDerivatives (CC-IGO BY-NC-ND 3.0 IGO) license (http://creativecommons.org/licenses/by-nc-nd/3.0/igo/ legalcode) and may be reproduced with attribution to the IDB and for any non-commercial purpose, as provided below. No derivative work is allowed. Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the IDB's name for any purpose other than for attribution, and the use of IDB's logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as part of this CC-IGO license. Following a peer review process, and with previous written consent by the Inter-American Development Bank (IDB), a revised version of this work may also be reproduced in any academic journal, including those indexed by the American Economic Association's EconLit, provided that the IDB is credited and that the author(s) receive no income from the publication. Therefore, the restriction to receive income from such publication shall only extend to the publication's author(s). With regard to such restriction, in case of any inconsistency between the Creative Commons IGO 3.0 Attribution-NonCommercial-NoDerivatives license and these statements, the latter shall prevail. Note that link provided above includes additional terms and conditions of the license. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Inter-American Development Bank, its Board of Directors, or the countries they represent. http://www.iadb.org 2016 1 Long-Term Impacts of Conditional Cash Transfers in Latin America: Review of the Evidence 1 Teresa Molina-Millan, Tania Barham, Karen Macours, John A. Maluccio, Marco Stampini Abstract We review the literature on the long-term impacts of Conditional Cash Transfer (CCT) programs in Latin America. Long-term impacts are defined as those that both: 1) are related to the accumulation of human capital, and; 2) are observed after beneficiary children have reached a later stage of the life-cycle. We focus on two life-cycle transitions. The first is children exposed to CCTs in utero or early childhood, who have then transitioned to school ages. The second is children exposed to CCTs during school ages, who have then transitioned to early adulthood. The evidence is inconclusive. The experimental literature finds consistent positive long-term effects on schooling, as well as some positive impacts on cognitive skills and learning, socioemotional skills and off-farm employment and income. However, many other estimates are not statistically different from zero and it is often not possible to discern whether this is due to lack of impact or to methodological shortcomings in the evaluation studies. Non-experimental evidence also is mixed. Developing further opportunities for analyses with rigorous identification strategies for the measurement of long-term impacts should be high on the research agenda. As original beneficiaries continue to age, this should also be increasingly possible. Keywords: Conditional Cash Transfers (CCTs), long-term impacts, Latin America, Progresa, Oportunidades, Prospera, Familias en Acción, Red de Protección Social. JEL code: I38. 1 Teresa Molina-Millan is Assistant Researcher at Nova School of Business and Economics ([email protected]); Tania Barham is Associate Professor of Economics and Faculty at the Institute of Behavioral Science, University of Colorado ([email protected]); Karen Macours is Associate Professor at the Paris School of Economics and INRA Researcher ([email protected]); John A. Maluccio is Associate Professor of Economics at Middlebury College ([email protected]); and Marco Stampini is Social Protection Lead Specialist in the Social Protection and Health division of the Inter-American Development Bank (IDB) ([email protected]). This work was supported by funds from the IDB Economic and Sector Work “CCT Long-Term Impacts: Literature Review and Research Opportunities” (RG-K1421). We thank Ferdinando Regalia, Hector Salazar, Norbert Schady, Pablo Ibarrarán, Caridad Araujo, Nadin Medellin, Pedro Cueva, an anonymous reviewer and the participants in an IDB seminar for useful comments and suggestions. All remaining errors are our own. This working paper was previously published as (and replaces) IDB technical note number 923. The content and findings of this paper reflect the opinions of the authors and not those of the IDB, its Board of Directors, or the countries they represent. 2 Resumen Revisamos la literatura sobre los impactos de largo plazo de los Programas de Transferencias Monetarias Condicionadas (PTMC) en América Latina. Definimos los impactos de largo plazo como aquellos que: (1) están relacionados con la acumulación de capital humano; y (2) se observan en una etapa posterior del ciclo de vida. Nos enfocamos en dos transiciones del ciclo de vida. La primera es aquella de los niños expuestos a los PTMC en el útero o en la primera infancia, quienes luego han transitado a la edad escolar. La segunda es aquella de los niños expuestos a los PTMC durante la edad escolar, quienes luego han transitado a la edad adulta. La evidencia no es concluyente. La literatura experimental encuentra efectos consistentemente positivos en escolaridad, así como algunos efectos positivos en el aprendizaje, las habilidades cognitivas, las habilidades socioemocionales y el ingreso proveniente de actividades no agrícolas. Sin embargo, otras estimaciones no son estadísticamente significativas; frecuentemente, no es posible discernir si esto se debe a la ausencia de impacto o a las debilidades metodológicas de los estudios. La evidencia no experimental también es mixta. El desarrollo de nuevas investigaciones para la medición de los impactos de largo plazo de los PTMC, basadas en estrategias de identificación rigurosas, debería ser una prioridad en la agenda de investigación. A medida que los beneficiarios originales crecen, implementar dichas investigaciones debería ser más sencillo. Palabras clave: Programas de Transferencias Monetarias Condicionadas (PTMC), impactos de largo plazo, América Latina, Progresa, Oportunidades, Prospera, Familias en Acción, Red de Protección Social. 3 1. Introduction Conditional Cash Transfer (CCT) programs, started in the late 1990s in Latin America, have become the anti-poverty program of choice in many developing countries in the region and elsewhere (Fiszbein and Schady 2009; Stampini and Tornarolli 2012; Levy and Schady 2013; Paes-Sousa, Regalia and Stampini 2013). Their objectives, including short-term poverty reduction via transfers, and long-term poverty reduction through enhanced investment in human capital, have broad policy appeal. The most common designs for CCTs generally follow the original design of Progresa, the Mexican program begun in 1997 (Levy 2006). Targeted to the poor, the principal program components include regular cash transfers to women (conditional on scheduled visits to health care providers for young children and on school enrollment and regular school attendance for school-age children) and social marketing to encourage investment in nutrition, health and education. Numerous evaluations of CCTs, many based on rigorous experimental designs, consistently show positive short-term impacts. These include poverty alleviation, improved health and nutrition, particularly for young children, and increased school attainment for older children (Annex 1). 2 In contrast, few studies investigate whether these short-term gains eventually translate into sustained long-term benefits. For example, does exposure to CCTs in early childhood lead to improvements in school-age outcomes? Do the increased investments in human capital improve labor market or other (young) adult outcomes? And, perhaps most importantly, do CCTs ultimately improve the welfare of the next generation? After nearly two decades of experience with these programs, there is a growing need, as well as an increasing potential, for establishing whether such long-term gains have been realized. We critically review the existing evidence on whether, and to what extent, CCT programs have begun to achieve their long-term objectives. To the best of our knowledge, there is no research that has examined whether individuals who benefitted from the interventions as children or teenagers remained poor as adults, or that has analyzed directly the effects on the welfare of the next generation. 3 This is likely because it is still too soon to investigate such next 2 Other recent work examining short-term educational outcomes of CCTs (among other programs) include reviews by Murnane and Ganimian (2014) and Glewwe and Muralidharan (2015) and meta-analyses by Saavedra and García (2012), Baird et al. (2013) and McEwan (2015). 3 There is some research examining longer-run poverty dynamics for CCT beneficiary households. For example, Gertler, Martinez and Rubio-Codina (2012) find that original beneficiary households in Mexico’s Progresa made investments that led to continued consumption gains more than five years after the start of the program, beyond those associated with ongoing CCT benefits. 4 generation outcomes, as even the earliest programs only began in the late 1990s. 4 Consequently, the bulk of the research we review focuses on whether CCTs have led to a sustainable accumulation of human capital. This is a key, and arguably necessary, component of their long-term objectives. Moreover, some elements of human capital, such as completed grades of schooling, are enduring and therefore can themselves be considered long-term outcomes. Rather than relying exclusively on time since first exposure, we define long-term impacts as those that both: 1) are related to the accumulation of human capital, and; 2) are observed after beneficiary children have reached a later stage of the life-cycle. We focus on two life-cycle transitions. The first is children exposed to CCTs in utero or early childhood (under age 6), who have transitioned to school ages. The second is children exposed to CCTs during school ages, who have transitioned to early adulthood, using age 18 as an approximate cut-off for adulthood. Short-term evidence makes clear that CCTs can lead to gains in nutrition and health for young children in the first group and to gains in schooling for the second. The duration of CCT exposure or the length of time since CCT exposure are not explicit criteria for selecting studies for our review. They are, however, important considerations for assessing impacts, and we document how they vary across studies. We restrict our review to research on Latin American programs, which have the longest history and continuous application, and consequently offer greater scope for analyzing long-term impacts. Moreover, the programs, while not identical, share many key design features, facilitating more general conclusions. The bulk of evidence comes from three countries, for which both experimental and nonexperimental evaluations on long-term effects are available: Mexico, Nicaragua and Colombia. Mexico’s Progresa 5 and Nicaragua’s Red de Protección Social (RPS) embedded experimental impact evaluations in their rollout, permitting experimental assessment of the differential impacts resulting from different program exposures over time. In both country cases, there were also surveys administered to similar, excluded populations, permitting assessment of absolute program effects using non-experimental techniques. Colombia’s national CCT, Familias en Acción had no experimental evaluation. For that CCT, we review a long-term non-experimental 4 Chile’s Subsidio Unitario Familiar, implemented since 1981, is sometimes considered the first CCT. However, this program did not include penalties for noncompliance with program conditions, nor did it have a rigorous ex-ante impact evaluation design (Fiszbein and Schady 2009). 5 While Mexico’s CCT program was renamed Oportunidades and then Prospera, for simplicity and because we are describing longterm impacts related to the initial stages of implementation, we refer to it throughout the paper as Progresa. 5 evaluation of the overall program, as well as non-experimental research that relies on secondary data on learning and tertiary-level educational outcomes. Experimental evidence for Colombia comes from the randomized evaluation of a distinct pilot CCT program in Bogotá. The remainder of the paper is organized as follows. In Section 2, we review the research that examines how participation in CCT programs for school-aged individuals (under age 18 years) translates into impacts on schooling, learning, labor market outcomes and marriage markets. Given the timeframe of the existing studies, evidence is necessarily limited to impacts during early adulthood, and hence reflects at least in part the trade-off between schooling and early work experience. 6 In Section 3, we review the research that examines how exposure in utero and early childhood translates into better cognitive, socioemotional, schooling and health outcomes during school ages. Section 4 concludes. 2. Exposure to a CCT during School Ages and Outcomes in Early Adulthood Nearly every Latin American CCT program that underwent a rigorous evaluation has been shown to have positive short-term impacts on school enrollment and attendance for children subject to schooling-related conditionalities, though the magnitudes of those impacts vary with program characteristics and type of target population (Fiszbein and Schady 2009 7 ; Murnane and Ganimian 2014; Glewwe and Muralidharan 2015). In many cases, positive results are also found for school progression. While these short-term impacts are encouraging, they fall short of providing definitive evidence on the more lasting changes that are the ultimate objective of CCTs. Does increased schooling in the short run lead to increased final educational attainment? And, perhaps more importantly, does increased schooling lead to better learning and improved labor market outcomes? A small number of studies based on programs in Mexico, Nicaragua and Colombia, provide evidence on these questions. For each country, we first describe the experimental 6 If CCTs increase the years of schooling, their beneficiaries will likely have less work experience than an otherwise similar cohort without program exposure. This may reduce the net returns from CCTs measured during early adulthood, particularly if the returns to work experience are diminishing (i.e., if the returns to the first few years of work experience, that only those who have not continued to study have, are relatively high). 7 Fiszbein and Schady (2009) review evidence from: Chile (Chile Solidario), Colombia (Familias en Acción), Ecuador (Bono de Desarrollo Humano), Honduras (Programa de Asignación Familiar-II), Jamaica (Programme of Advancement through Health and Education), Mexico (Progresa), Nicaragua (Atención a Crisis and Red de Protección Social), Bangladesh (Female Secondary School Assistance Program), Cambodia (Japan Fund for Poverty Reduction and Cambodia Education Sector Support Project), Pakistan (Punjab Education Sector Reform Program) and Turkey (Social Risk Mitigation Project). 12 A final consideration for the whole set of studies pertains to the difficulty of studying young adults who are still undergoing important life cycle transitions. For example, the observed delayed entry in the labor market for males means that for this age group the evaluation can reveal only initial, or partial, information about the ultimate program effects on occupation or income. The obvious solution, of course outside the scope of these specific studies, is to continue following these individuals further into the future until all of them have left school and entered the labor market, while making sure to keep attrition as low as possible. Multiple measurements, so that time paths of effects could be traced out, would be even more valuable. 2.4 Nicaragua: Experimental Evidence Modeled after Progresa, RPS started in 2000, and had a short-term experimental evaluation built into its initial stages. This evaluation took place in 42 localities in six rural municipalities with initial poverty rates around 80 percent. Unlike Mexico’s Progresa and Colombia’s Familias en Acción, RPS was not a program with national coverage. Localities were randomly assigned to earlyand late-treatment groups in a public lottery. The 21 early-treatment localities became eligible for transfers in November 2000 and were eligible for three years, receiving their last transfers in late 2003. The 21 late-treatment localities were phased in at the beginning of 2003 and were also eligible for three years of transfers. Households in the early-treatment group did not receive any transfers after 2003, and had no conditionalities after that date. However, they continued to be eligible to use the RPS-provided health supply services. The program ended in late 2005. As in Mexico, this difference in the timing of exposure, between randomly allocated “early-” and “late-treatment” groups is the cornerstone of the long-term experimental evaluation in Nicaragua, and provides differential, rather than absolute, estimated effects (Barham, Macours and Maluccio 2016). In contrast to Mexico, however, it compares groups that randomly received the program for a fixed 3-year period at different points in time. In Mexico, the relative difference in length of randomly generated exposure diminishes with time since exposure in both earlyand late-treatment groups continues to increase (as the program is ongoing). This is not the case in Nicaragua. Between November 2009 and November 2011, i.e., approximately 10 years after the start of the program for the early-treatment group, a long-term follow-up evaluation survey was conducted (Barham, Macours and Maluccio, 2013, 2016). All original households were tracked. When a household was found (and interviewed), but a former member 12 or under in 2000 was no longer resident, the individual was tracked and his or her (new) household added to the 13 sample. Relative to other long-term evaluations, attrition was low in Nicaragua. Households and individuals in the original sample were tracked beyond their original communities throughout Nicaragua and into Costa Rica, leading to a household attrition rate below 8 percent. Attrition was higher for individual-level outcomes, especially among the more mobile young adults who were not always possible to interview in person (but for whom some proxy information could be collected, as in Mexico). For example, for males 9-12 years old in 2000, attrition in 2010 was 6 percent for schooling and basic occupation information, 12 percent for earnings data, and 19 percent for test scores. Even with substantial resources dedicated to tracking respondents in this ten-year panel, keeping attrition to a minimum, the remaining attrition is unlikely to be random; indeed MolinaMillan and Macours (2015) demonstrate that there is remaining attrition bias in the standard intent-to-treat estimates, and suggest a correction that uses information from the intensive tracking carried out during the field work to re-weight and correct for sample selection. The 2010 follow-up survey, together with pre-intervention data, is used to estimate the differential impacts of RPS on educational attainment, learning and labor market outcomes for males who were 9-12 years old in 2000 (and therefore 18-21 at follow-up). Due to the random difference in the timing of the interventions for the earlyand late-treatment groups, focusing on this specific age cohort allows estimating the long-term effects of benefiting from a CCT in a period of the life cycle that is considered critical for educational investments (the age at which the probability of dropping out of school is high) versus three years later (at which point many would have already dropped out). All estimates are weighted for attrition as described in MolinaMillan and Macours (2016). Barham, Macours and Maluccio (2016) show that the short-term program effect of a half grade increase in completed schooling for males was largely sustained five years after the end of the program and into early adulthood. In contrast to the findings from the long-term evaluation in Mexico, the differential increase in grades attained was accompanied by gains in learning. Males in the early-treatment group experienced an average improvement of about 0.2 standard deviations on standardized tests in mathematics and Spanish (relative to the late-treatment group). The effect size was similar across the range of achievement tests. The study also found positive differential impacts of about 0.2 standard deviations on socioemotional outcomes, such as optimism and positive self-evaluation. Last, it finds differential effects on labor market outcomes, with the young men in the early-treatment group being more likely to work off-farm, 14 migrating temporarily to do so. This results in an increase of 10-30 percent in monthly off-farm income. Overall, this study shows that the RPS produced large long-term differential impacts on earnings for men, consistent with increased human capital leading to better labor market outcomes. 12 2.5 Nicaragua: Non-Experimental Evidence Barham, Macours and Maluccio (2016) also explore the absolute effects of exposure to three years of RPS on outcomes measured 10 years after the start of the program in the early treatment group, and 7 years after in the late-treatment group. As for Mexico, these estimates reflect different underlying parameters than the differential effects estimated using the experimental variation. The non-experimental control group is drawn from a sample of individuals living in 21 localities in adjacent municipalities, which were selected using the same marginality index used for selecting localities in the experimental evaluation. This sample was first surveyed in 2002, i.e., two years after the start of the program, as part of the short-term program evaluation. Individuals from the earlyand, separately, late-treatment groups are matched to individuals in the described non-experimental control sample, based on a set of individual, household, and community characteristics measured in 2000 and 2002, including the marginality index used for selecting localities. Then, outcomes in 2010 are compared to measure program impacts. The key assumption underlying this strategy is that, given the selection of similar and neighboring localities, the matching on observables also controls for all other differences in unobservables. The non-experimental results show positive absolute impacts on schooling, learning, labor market outcomes and earnings for young men 9-12 in 2000, in line with the experimental results but generally larger in magnitude. For example, the absolute effect on grades attained is more than a full year of school in the early-treatment group. For women in the same age group, although there were no significant experimental differential effects on grades attained, the standardized tests or income, the non-experimental results demonstrate significant absolute learning effects of about 0.3 standard deviations that are similar in magnitude across the early12 Parallel analyses for the same age cohort of women, however, showed no significant differential effects, possibly because the program had similar effects on this cohort of girls in the earlyand late-treatment groups (as girls tended to drop out of school at older ages). See section 2.6. 15 and late-treatment groups. This suggests that positive and equal absolute impacts on learning may underlie the lack of significant experimental results (which capture the differential effect) for females. This is further supported by the finding that there was a large experimental differential impact on grades attained for an older cohort of girls, aged 13-14 in 2000. 2.6 Nicaragua: Assessment of the Evidence Many of the concerns detailed above in the assessment of the evidence for Mexico are also pertinent to Nicaragua (and Colombia below). For brevity we make shorter reference to them here, though they are not necessarily less important. Despite attrition rates that are much lower than in other similarly long-term studies, attrition bias remains a source of concern, especially for certain age groups. The validity of the estimates still depends on the validity of assumptions made to re-weight the data to correct for sample selection and the quality of data provided by proxy informants. While Barham, Macours, and Maluccio (2016) indicate that estimates are robust to alternative assumptions, suggesting that they are reliable, it is not possible to determine the direction of any remaining bias. The fact that RPS was not a national program and targeted areas in which pre-treatment levels of schooling were very low, implies that results may have limited external validity for many other settings in Latin America. In other words, it is unclear whether the same results on learning and earnings would be seen in other, less poor, regions or countries. As in the studies on Mexico, beneficiaries were only observed as young adults, when some were still studying (despite the average lower levels of education) and many were still living with their parents. Hence, only with additional rounds of data collection will analysts be able to gauge the full long-term impact of RPS in terms of returns to human capital and, more broadly, welfare outcomes. Moreover, the experimental results on learning and labor markets are only significant for the boys, and while results for an older cohort of girls indicate there is a large differential impact on their grades attained, achievement tests were not administered to them so their learning results could not be examined. For the non-experimental results, the same caveats apply as for the non-experimental matching results for studies on Mexico, although the details vary. 16 2.7 Colombia: Experimental Evidence Barrera-Osorio, Linden and Saavedra (2015) provide experimental evidence on the long-term impacts of a one-year CCT pilot, Subsidios, implemented in 2005 in Bogotá. 13 The program consisted of a conditional education transfer for secondary school students (aged 12-18), and included ages at which the probability of dropout is high in Colombia. The authors tested three alternative program designs. The first is referred to as the “basic treatment”, in which transfers were made every other month, conditional on meeting a specified attendance target. The second is a "savings treatment", in which one third of the transfer payments was delayed until enrollment in the following school year. The third is a "tertiary treatment", implemented only for students in upper secondary school (grades 9-11), in which one third of the transfer payments was delayed until after graduation from secondary school and then paid either: 1) upon enrollment in a tertiary institution if the individual enrolled in one, or; 2) one year later, if the individual did not enroll. All those in the tertiary treatment who graduated from secondary school (including if this happened in the years following the end of the pilot) became eligible for this final transfer. The basic and savings treatments are compared relative to one randomized control group (comprising secondary school students aged 12-18). Separately, the tertiary treatment is compared to a different randomized control group (comprising only students in upper secondary school at baseline, to match the treatment group for this intervention). 14 The three treatment arms were randomly assigned at the individual level. The control groups never received the intervention, allowing the authors to evaluate the long-term absolute effect of this program. The authors merge program participation data with national administrative records on upper secondary school graduation exams and enrollment in tertiary institutions for the eight years following the 2005 intervention. These merges were based on student ID number, full 13 This is the only CCT program included in our review that operated exclusively in urban areas and that had no conditionalities related to younger children. It is described further in Barrera et al. (2011). 14 The savings treatment implied a reduction of the monthly transfer amount, which could lead to children attending school less if families faced short-term liquidity constraints. In contrast, if long-term savings constraints were more significant than immediate liquidity constraints, the savings treatment could generate higher re-enrollment rates than the basic treatment, as the families received the money when the next year’s schooling expenses began, without affecting current attendance rates. The tertiary treatment generated incentives for graduation, which could result in higher re-enrollment and graduation rates and higher levels of enrollment in tertiary institutions, unless yearly saving constraints prior to reaching tertiary levels were the most binding constraint. Attendance under the tertiary treatment also could increase if participants viewed school attendance as instrumental for graduation (Barrera-Osorio, Linden and Saavedra 2015). 17 name and date of birth. The study shows that the percentage that could be merged corresponds to expected rates of graduation from upper secondary school and enrollment in tertiary institutions for the study population, and that the probability of a successful merge was not related to baseline characteristics, nor did it differ between basic treatment and control groups. Results for students in grades 9 to 11 (i.e., in upper secondary education) at the time of program exposure show that only the savings treatment significantly increased the probability of taking the secondary school exit exam, by 2.8 percentage points (about 4 percent). However, it is not possible to reject the hypothesis that the different experimental treatments had equal impacts. Both the savings and tertiary treatments led to higher enrollment in tertiary institutions (by about 10 and 20 percent, respectively) suggesting that savings constraints may have been a barrier for enrollment in tertiary education. Furthermore, the savings treatment encouraged enrollment in universities, while the tertiary treatment led to enrollment in lower quality tertiary institutions. This suggests that the incentives for tertiary enrollment encouraged students to enroll more indiscriminately. No significant treatment effects were found for students first exposed in lower secondary education (grades 6 to 8). These results broadly confirm short-term impacts estimated one year after the intervention for individuals in grade 11 at the start of the program, and thus on the cusp of the post-secondary school transition (Barrera-Osorio et al. 2011). Using self-reported information, the authors found a significant increase in the probability of enrolling in a tertiary institution among those assigned to the savings (9.4 percentage points) or tertiary (48.9 percentage points) treatments in grade 11, but no significant effects on graduation rates for any of the treatment arms. For these same individuals, the program had a (corresponding) significant negative impact on labor force participation (reducing the probability that the primary activity was work and reducing hours worked and earnings from work in the last week) for those in the tertiary treatment. 2.8 Colombia: Non-Experimental Evidence Colombia’s national CCT program, Familias en Acción, did not have an experimental impact evaluation but was the subject of two long-term non-experimental evaluations. García et al. (2012) conduct a comprehensive non-experimental evaluation of long-term effects on education and labor market outcomes. The identification strategy relies on a simpledifference (with baseline controls) or difference-in-difference estimation and compares children 18 from eligible households from municipalities covered by the program in 2002 with children from potentially eligible households from comparable areas that were only targeted in 2007. These control municipalities were selected based on similarities with the treatment in terms of region, level of urbanization, number of eligible households, a quality of life index, and health and school infrastructure. The authors investigate differential exposure effects, using data from 2002 (i.e. pre-intervention), and a follow-up survey conducted between November 2011 and February 2012. They measure these differential impacts on a set of education and labor market outcomes for which the impact of the program can be thought of as cumulative. Examining young adults aged 18-26 in 2012, the study found that the 2 to 5 years additional exposure to Familias en Acción increased school attainment by 0.6 grades in rural areas. Also for rural areas, it found a positive significant impact on the probability of graduating from upper secondary school, alongside a negative impact on the probability of enrolling in tertiary education, which is somewhat puzzling. The only significant impact on labor market outcomes found by the study was an increase of 2.5 percentage points in the probability of formal employment among women in rural areas. In urban areas, impact estimates for this age group on these same outcomes are not significantly different from zero. The study further reports estimates of impacts on cognitive skills, based on a mathematics ability test and the Raven’s Progressive Matrices test, for adolescents aged 12-17 in 2012. It shows that the differential exposure to Familias en Acción increased mathematics scores by 1.07 standard deviations, which is quite large compared to other studies on learning outcomes. 15 As this result is for the cohort that was 2-7 years old at baseline, it reflects in part the effect of exposure to the CCT program during early childhood. Baez and Camacho (2011) investigate the effects of up to nine years of participation in Familias en Acción using household survey data, registration records from SISBEN 16 and administrative data on the results of the secondary school graduation test, the same data source used by Barrera-Osorio, Linden and Saavedra (2015). The study uses both differencein-difference matching and Regression Discontinuity Design (RDD). It focuses on two different samples of children who had the potential to complete grade 11 by 2009, constructed from the 2002 baseline program evaluation sample (for the matching analysis) and from merging the program administrative data with the SISBEN census (for the RDD). These two samples were 15 Results for the Raven (+0.16 standard deviations) were not statistically significant. 16 The SISBEN is Colombia’s proxy means score. Eligibility for Familias en Acción is based on having a SISBEN score below a given threshold. 19 merged with the national secondary school graduation test scores based on national ID number, full name and date of birth. The results show that beneficiary children were 4-8 percentage points more likely than non-beneficiary children to complete secondary school. However, the authors found no evidence of differential secondary school exit test score performance, conditional on completion. 2.9 Colombia: Assessment of the Evidence Overall, Barrera-Osorio, Linden and Saavedra (2015) provide a good example of the possibilities and the limits of using secondary administrative data to follow up on an earlier experimental evaluation. If data can be merged successfully (which notably requires excellent unique identifiers at baseline), it is possible to conduct a study with low budget and high internal validity and statistical power. Match rates from merging the different data sources were relatively high (e.g., 70 percent or above for upper secondary school exit exams) and largely unrelated to treatment arm. On the other hand, the set of outcomes that can be studied using such secondary data, and hence the possibility to understand the different parts of the impact pathway, are naturally more limited. The results in Baez and Camacho (2011) are subject to similar limitations as BarreraOsorio, Linden and Saavedra (2015). In the former, however, there may be substantially more selection bias as match rates of the merge between program administrative records and school test data are less than 25 percent. In addition, the finding on test scores is not generalizable to the complete population covered by Familias en Acción as the test was only administered to children who stayed in school and progressed until grade 11, which may introduce further sample selection bias. An important drawback of the use of the RDD approach in Colombia is that the SISBEN is not only used to determine eligibility for Familias en Acción, but also for several other government interventions (Velez et al. 1999). Hence the estimates are potentially confounding the impacts of different programs. Finally, as they acknowledge, the study design cannot distinguish the effect for different ages from the effect due to length of exposure. For example, enrolled beneficiaries joining the program when they are older, have fewer years of school to complete than younger beneficiaries, thus they are more likely to be observed finishing secondary school. If this is the case, shorter exposure to the program could be incorrectly associated with higher secondary school completion rates. 20 The concerns regarding simple difference or difference-in-difference estimates in García et al. (2012) are similar to those described for the non-experimental studies for Mexico and Nicaragua. The study relies on the arguably strong assumption that selection into the program is only related to observable and time-invariant (in the case of difference-in-differences) unobservable characteristics. As the studied outcomes are typically only meaningfully observed for adults (secondary school completion) or likely should be interpreted differently for adults compared to children (e.g., employment), controlling for baseline outcomes might not adequately control for unobservable confounders. In addition, the baseline survey used in the evaluation of Familias en Acción was implemented after the program had already been announced, and as such might reflect program-related changes in behavior, or anticipation effects. 3. Exposure to a CCT during Early Childhood and Outcomes during School Age In this section we review the research that examines whether and how exposure to CCT programs in utero and under age 6 translates into better cognitive, socio-emotional and health outcomes during primary and secondary school. The evidence base for this cohort is somewhat more limited. As above, we draw on long-term evidence from Mexico, Nicaragua and Colombia. 3.1 Mexico Fernald, Gertler and Neufield (2009) investigate the effect of Oportunidades on child growth, cognition, language and behaviors, 10 years after the start of the program. They exploit the 18-month differential exposure between the earlyand lateexperimental treatment localities described above for the school-age cohorts examined by Behrman, Parker and Todd (2009a). Outcomes are measured in 2007 for individuals who were in utero or under 13 months of age when the program started, and therefore 8-10 years old at the time of their follow-up survey. The study found a significant reduction in behavioral problems. No significant experimental results are found for any measure of child growth, cognition or language when using the main experimental approach (comparison of mean outcomes in earlyand latetreatments). The authors also present an alternative estimate, however, which takes into account the potential cumulative cash transfers received between initial household enrollment 21 and 2007. Potential cumulative transfers differ across households for two reasons: 1) the experimental variation in timing of entry into the program, and; 2) different household composition and grade achievements of potential eligible children at baseline, since transfer amounts are tied to gender, age and grade-level. Incorporating this latter source of nonexperimental variation, the authors report a negative association between accumulated cash transfers and the number of reported behavioral problems, consistent with the findings from the experimental evaluation. In addition, they find that higher cash transfers are significantly and positively associated with height-for-age z-scores and higher verbal and cognitive test scores. Because the accumulated cash transfers depend on household structure as well as the randomized assignment, the interpretation and internal validity of these results have been called into question (Attanasio, Meghir and Schady 2010). Given the lack of evidence when only using the randomized assignment, the results must be driven by differences in baseline household demographics. Yet baseline household demographics are not randomly assigned, and might well affect cognitive and anthropometric outcomes in their own right. Behrman, Parker and Todd (2009b) examine children aged 0-8 at the start of the program, and thus 6-14 in 2003, using first-difference and difference-in-difference estimators exploiting the original experiment, and also difference-in-difference non-experimental matching methodologies to estimate absolute program effects as in their work with the older cohorts. Because of their ages at baseline, however, all but the oldest of these children do not have meaningful baseline outcomes regarding schooling. They modify the difference-in-difference estimator to instead control for the outcomes of children who were 6-14 at baseline. Attrition for the targeted age group is lower than for the older cohort, at 20 percent for eligible children, and similarly addressed through re-weighting. The authors find a slight differential reduction (of 0.05 years) in the age of entry into primary school for girls 7-8 years old in 2003, but no significant effects for the older ages or for boys. They also examine program impacts on grade progression, which reflects both enrollment and continuation. The results show that the 18month differential exposure to Progresa had no significant effect on grade progression for children aged 9-11 in 2003. 17 In contrast, the difference-in-difference matching estimates that compare the original treatment group receiving six years of benefits to the 2003 nonexperimental comparison group show positive and significant absolute effects in progression rates of about 15 percent for boys and 7 percent for girls. 17 Results are not available for the youngest age group. 28 Fernald, L.C., Gertler, P.J. and Neufeld, L.M. (2009). “10-year Effect of Oportunidades, Mexico's Conditional Cash Transfer Programme, on Child Growth, Cognition, Language, and Behaviour: a Longitudinal Follow-up Study.” The Lancet 374(9706), 1997-2005. Fernald, L.C.H. and Hidrobo, M. (2011). “Effect of Ecuador’s Cash Transfer Program (Bono de Desarrollo Humano) on Child Development in Infants and Toddlers: a Randomized Effectiveness Trial.” Social Science and Medicine (1982), 72(9), 1437–46. doi:10.1016/j.socscimed.2011.03.005 Filmer, D. and Schady, N. (2014). “The Medium-Term Effects of Scholarships in a Low-Income Country.” Journal of Human Resources, 49(3), 663–694. doi:10.1353/jhr.2014.0022 Fiszbein, A. and Schady, N.R. (2009). “Conditional Cash Transfers: Reducing Present and Future Poverty.” World Bank Policy Research Report. Washington, DC, United States: 24, World Bank. Gaarder, M., Glassman, A. and Todd, J. (2010). “Conditional Cash Transfers and Health: Unpacking the Causal Chain.” Journal of Development Effectiveness, 2(1), 6–50. Galiani, S. and McEwan, P.J. (2013). “The Heterogeneous Impact of Conditional Cash Transfers.” Journal of Public Economics, 103, 85–96. doi:10.1016/j.jpubeco.2013.04.004 García, A., Romero, O.L., Attanasio, O. and Pellerano, L. (2012). “Impactos de Largo Plazo del Programa Familias en Acción en Municipios de Menos de 100 mil Habitantes en los Aspectos Claves del Desarrollo del Capital Humano.” Technical report, Union Temporal Econometria S.A. SEI. con la asesoria del IFS. Gertler, P., Martinez, S. and Rubio-Codina, M. (2012). "Investing Cash Transfers to Raise Long Term Living Standards." American Economic Journal: Applied Economics 4(1): 164–92 IDB (2014). “Conditional Cash Transfers Research Brief.” Unpublished manuscript. Washington, DC, United States: Inter-American Development Bank. Glewwe, P., and K. Muralidharan, K. (2015). Improving School Education Outcomes in Developing Countries: Evidence, Knowledge Gaps, and Policy Implications. Research on Improving Systems of Education Working paper, RISE-WP-15/001. Kling, J.R., Liebman, J.B. and Katz, L.F. (2007). "Experimental Analysis of Neighborhood Effects." Econometrica 75(1): 83-119. Levy, S. (2006). “Progress Against Poverty: Sustaining Mexico’s Progresa-Oportunidades Program.” Washington DC, United States: Brookings Institution Press. Levy, Santiago, and Schady, N. (2013). “Latin America’s Social Policy Challenge: Education, Social Insurance, Redistribution,” Journal of Economic Perspectives, vol. 27(2): 193-218. Macours, K., Schady, N. and Vakis, R. (2012). “Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment.” American Economic Journal: Applied Economics (4)2. Maluccio, J.A., and Flores, R. (2005). “Impact Evaluation of a Conditional Cash Transfer Program: The Nicaraguan Red de Protección Social.” Research Report No. 141, International Food Policy Research Institute. McEwan, P. (2015). “Improving Learning in Primary Schools of Developing Countries: A MetaAnalysis of Randomized Experiments.” Review of Educational Research, vol. 85(3): 35394. 29 Molina-Millan, T. and Macours, K. (2016). “Attrition in Randomized Control Trials: Regular versus Intense Tracking Protocols.” Unpublished manuscript. Murnane, R.J. and Ganimian, A.J. (2014). “Improving Educational Outcomes in Developing Countries: Lessons from Rigorous Evaluations”, NBER Working Paper 20284. Paes-Sousa, R., Regalia, F. and Stampini, M. (2013). “Conditions for Success in Implementing CCT Programs: Lessons for Asia from Latin America and the Caribbean.” IDB Policy Brief n. 185. Washington D.C., USA: Inter-American Development Bank. Parker, S., Rubalcava, L., Teruel, G. (2012). “Do Conditional Cash Transfer Programs Improve Work and Earnings among its Youth Beneficiaries? Evidence after a Decade of a Mexican Cash Transfer Program.” Unpublished manuscript. Rackstraw, E. (2014). “A Decade Later: An Evaluation of the Longer-Term Impacts of a Honduran Conditional Cash Transfer.” Wellesley College Honors Thesis Collection, Paper 215. Rasella, D., Aquino, R., Santos, C.A.T., Paes-Sousa, R., and Barreto, M.L. (2013). “Effect of a Conditional Cash Transfer Programme on Childhood Mortality: a Nationwide Analysis of Brazilian Municipalities.” Lancet, 382(9886), 57–64. doi:10.1016/S0140-6736(13)60715-1 Rodriguez-Oreggia, E. and Freije, S. (2012). “Long Term Impact of a Cash Transfers Program on Labor Outcomes of the Rural Youth in Mexico.” CID Working Paper No 230. Harvard University. Ruiz-Arranz, M., Davis, B., Handa, S., Stampini, M. and Winters, P. (2006). “Program Conditionality and Food Security: The impact of PROGRESA and PROCAMPO Transfers in Rural Mexico.” Economia, 7(2), 249–278. Saavedra, J.E. and García, S. (2012). “Impacts of Conditional Cash Transfer Programs on Educational Outcomes in Developing Countries.” RAND Corporation. Working Papers, (WR-921-1). Retrieved from http://www.rand.org/pubs/working_papers/WR921-1.html Schultz, T.P. (1997). “School Subsidies for the Poor: Evaluating the Mexican Progresa poverty program.” Journal of Development Economics, 74: 199-250. Stampini, M. and Tornarolli, L. (2012). “The Growth of Conditional Cash Transfers in Latin America and the Caribbean: Did They Go Too Far?” IDB Policy Brief n. 185. Washington D.C., USA: Inter-American Development Bank. Velez, C. E., Castano, E., and Deutsch, R. (1999). "An Economic Interpretation of Targeting Systems for Social Programs: the case of Colombia’s SISBEN." World Bank. Working Papers. 30 Annex 1 – Short term impacts of CCTs on the children in beneficiary households Consumption and Poverty. Various impact evaluations, summarized in Fiszbein and Schady (2009), show that CCTs have unambiguously met their primary short-run objective of increasing consumption and reducing poverty. Decomposition exercises, like those carried out by Stampini and Tornarollli (2012) and Levy and Schady (2013) also suggest that CCTs have been important in reducing poverty and inequality in the region in the last decade. CCT have not only increased consumption, they have also improved its composition. For example, Ruiz-Arranz et al. (2006) show that CCTs have increased the quantity, quality and variety of food intake, leading to consumption of more nutritious and expensive goods such as meat and vegetables. Education. Another consistent finding across rigorous impact evaluations is that CCTs have substantially decreased child labor (Galiani and McEwan, 2013; Levy, 2006; Edmonds and Schady, 2012) and increased school enrollment and attendance (with rates that vary from 0.5 percentage points (pp) in Jamaica to 12.8 pp in Nicaragua). This is –together with higher use of health servicesthe key behavioral outcome intended by the theory of change of CCTs. CCTs also increased school attainment. For example, in Mexico after 3-5 years of participation in Oportunidades, the beneficiaries accumulated between ½ and 1 year of additional schooling. The evidence on learning achievement is mixed (Fiszbein and Schady, 2009; García et al., 2012; Saavedra and García, 2012). Barham et al. (2016) find that, in Nicaragua, receiving the CCT for three years had significant impacts on years of schooling and on mathematics and language learning for young men 10 years after participating in the program. Learning increased by ¼ of a standard deviation, which loosely corresponds to half a year of learning. On the other hand, Behrman, Parker and Todd (2009b) find that higher enrollment levels have not resulted in better performance on achievement tests in Mexico. Evidence from outside the region is also mixed. Baird et al. (2011) report positive impacts on learning for a pilot CCT in Malawi, while Filmer and Schady (2014) and Benhassine et al. (2015) find no effect of a CCT on learning outcomes in Cambodia and Morocco, respectively. From the perspective of CCTs, increasing schooling is the relevant indicator and main responsibility. At the same time, from a social perspective, learning is the purpose of schooling and the possibility that some children that go to school –whether they are CCT beneficiaries or notmay not learn is a reason for concern. The mixed findings stress the need to seriously address the issue of quantity and quality of education services, and how this interacts with the demand-side subsidies provided by CCTs. 31 Health. CCT programs have consistently shown positive effects on the use of preventive health services. Estimated impacts range between 6.3 pp in Nicaragua and 33 pp in Colombia. Some evaluations have also found that CCTs contributed to improvements in child height among some population groups, although the evidence is mixed (Fiszbein and Schady, 2009). There is some evidence that program beneficiaries have better health status and reduced morbidity (Gaarder et al., 2010). Rasella et al. (2013) report that Bolsa Família reduced infant mortality caused by conditions associated with poverty, such as malnutrition and diarrhea. As is the case for education, health outcomes depend largely on the quality of health services, an issue that lies outside of CCTs’ direct responsibility. Child development. Rigorous evaluations suggest that CCT programs have positive impacts on child development. Paxson and Schady (2010) study the Bono de Desarrollo Humano (BDH) in Ecuador, and find significant effects on somewhat older children, although only for those in the poorest wealth quintile. Fernald and Hidrobo (2011) also study the BDH and find that children randomized to receive in Ecuador had higher scores on a test of the number of words children can say. Macours et al. (2012) find that the Atención a Crisis program in Nicaragua had an effect of about 0.12 standard deviations on the family of cognitive and behavioral outcomes they analyze. Source: IDB (2014, p. 2-4).