Impact evaluation of a cluster program: An application of synthetic control methods

Aboal, Diego; Crespi, Gustavo; Perera, Marcelo Working Paper Impact evaluation of a cluster program: An application of synthetic control methods IDB Working Paper Series, No. IDB-WP-836 Provided in Cooperation with: Inter-American Development Bank (IDB), Washington, DC Suggested Citation: Aboal, Diego; Crespi, Gustavo; Perera, Marcelo (2017) : Impact evaluation of a cluster program: An application of synthetic control methods, IDB Working Paper Series, No. IDBWP-836, Inter-American Development Bank (IDB), Washington, DC, https://doi.org/10.18235/0000810 This Version is available at: https://hdl.handle.net/10419/173889 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by-nc-nd/3.0/igo/legalcode IDB WORKING PAPER SERIES Nº IDB-WP-836 Impact Evaluation of a Cluster Program An Application of Synthetic Control Methods Diego Aboal Gustavo Crespi Marcelo Perera Inter-American Development Bank Institutions for Development Sector September 2017 September 2017 Impact Evaluation of a Cluster Program An Application of Synthetic Control Methods Diego Aboal* Gustavo Crespi** Marcelo Perera* *CINVE ** Inter-American Development Bank Cataloging-in-Publication data provided by the Inter-American Development Bank Felipe Herrera Library Aboal, Diego. Impact evaluation of a cluster program: an application of synthetic control methods / Diego Aboal, Gustavo Crespi, Marcelo Perera. p. cm. — (IDB Working Paper Series ; 836) Includes bibliographic references. 1. Tourism-Uruguay. 2. Industrial clusters-Uruguay. 3. TourismGovernment policy-Uruguay-Mathematical models. I. Crespi, Gustavo. II. Perera, Marcelo. III. Inter-American Development Bank. Competitiveness, Technology and Innovation Division. IV. Title. V. Series. IDB-WP-836 Copyright © Inter-American Development Bank. This work is licensed under a Creative Commons IGO 3.0 AttributionNonCommercial-NoDerivatives (CC-IGO BY-NC-ND 3.0 IGO) license (http://creativecommons.org/licenses/by-nc-nd/3.0/igo/ legalcode) and may be reproduced with attribution to the IDB and for any non-commercial purpose, as provided below. No derivative work is allowed. Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the IDB's name for any purpose other than for attribution, and the use of IDB's logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as part of this CC-IGO license. Following a peer review process, and with previous written consent by the Inter-American Development Bank (IDB), a revised version of this work may also be reproduced in any academic journal, including those indexed by the American Economic Association's EconLit, provided that the IDB is credited and that the author(s) receive no income from the publication. Therefore, the restriction to receive income from such publication shall only extend to the publication's author(s). With regard to such restriction, in case of any inconsistency between the Creative Commons IGO 3.0 Attribution-NonCommercial-NoDerivatives license and these statements, the latter shall prevail. Note that link provided above includes additional terms and conditions of the license. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Inter-American Development Bank, its Board of Directors, or the countries they represent. http://www.iadb.org 2017 Abstract* This paper analyzes the impact of a cluster tourism policy in the region of Colonia, Uruguay. The policy was implemented as part of an IDBsupported program. The study conducted a comparative analysis of Colonia vs. other tourism regions of the country applying a synthetic control method. This method of identifying the counterfactual is especially useful in comparative case studies where there are a limited number of control units. This is the first time that the synthetic control method has been applied to a cluster policy. The estimations show a positive impact of the cluster program on the inflow of international tourists to Colonia of 30 percent in the period 2008–2015; however, no significant impact on total expenditure was found. JEL Codes: H43, O25, O54, R10 Keywords: cluster policy, impact evaluation, synthetic control methods * The authors are grateful to the Inter-American Development Bank for its financial support and to Agustin Barboza for his research assistance. They wish to acknowledge the valuable comments and suggestions of Philip Keefer, Rodolfo Stucchi, other participants at the 10th meeting of the LACEA’s Impact Evaluation Network in Washington DC, and CINVE’s internal seminar participants. 2 1. Introduction Since the publication of the seminal work by Porter (1990), cluster development policies (CDPs) have become increasingly popular as a tool for promoting productive development in developed and developing economies alike.1 Although the scope and size of CPD interventions vary across countries, they frequently operate at the subnational level, where firms tend to agglomerate around specialized productive activities. The aim of these policies is to eliminate, or at least compensate for, coordination failures among firms and between firms and governments to guarantee the provision of the public goods needed to ensure the competitiveness of the agglomeration. Despite their pervasiveness, CDPs are among the least evaluated productive development policies. Evaluating the impact of CDP interventions is far more complex than evaluating a typical productive development policy. Most of the literature on impact evaluation of productive development policies draws on the social policy approach, where the focus of intervention is on lifting individual beneficiaries out of poverty. However, evaluating the impact of a CDP on individual firms does not provide useful information. Since the aim of CDP programs is coordination between private and public actors to provide public goods—which by definition affect all stakeholders of clusters—all of the firms in the agglomeration are to some extent treated firms. Some are treated directly because they actively engage in cluster activities, while others are treated indirectly as the result of the provision of public goods or spillovers. Thus, these policies should be analyzed at a higher level of aggregation, comparing treated agglomerations with untreated ones. This paper takes a step forward in this direction by assessing the impact of a CDP program in a particular region of Uruguay and using the synthetic control method to build a control group comprising other untreated regions in the same country. The paper is structured as follows. Section 2 presents a brief review of the literature on CDP programs and evaluations. Section 3 describes the Program for the Competitiveness of Clusters and Production Chains (Programa de Apoyo a la Competitividad de Conglomerados, or PACC) and outlines its impact channels. Section 4 describes the data. Section 5 presents the empirical strategy. Section 6 summarizes the results, and Section 7 concludes. 1 A survey carried out by the European Cluster Observatory in 2012 identified about 570 “cluster initiatives” across the European Union. In 2010, the U. S. Small Business Administration launched some 40 cluster programs across the country. Similar large-scale initiatives are found in India and China (IDB, 2014). 3 2. Literature Review The study of agglomeration economies can be traced back to Marshall (1920). It was later expanded in Arrow (1962) and Romer (1986), and formalized by Glaeser et al. (1992) as the Marshall-Arrow-Romer (MAR) model. Agglomeration economies are usually defined as the formation of clusters of firms that belong to a specific industry and are located in a particular geographic area. The tendency of firms in the same industry to concentrate geographically has been extensively studied in the literature (Delgado, Porter, and Stern, 2014; Ellison and Glaeser, 1997; Jaffe, Trajtenberg, and Henderson, 1993; Kerr and Kominers, 2015). Agglomeration helps firms establish links with other firms within the cluster, which leads to gains from coordination and the internalization of externalities at the cluster level. However, coordination failures are a common problem, leading to sub-optimal allocation of resources. As Rosenstein-Rodan (1943) points out, coordination failures frequently emerge when the investment decision of one agent is interrelated to those of others and externalities emerge due to this interrelationship. These coordination failures have particularly adverse consequences for the provision of cluster-specific public goods.2 Once these failures have been eliminated, the theory says, linkages between the firms will become stronger. These stronger linkages will build trust and foster the kinds of knowledge spillovers that tend to arise in every market transaction. Furthermore, Maffioli, Petrobelli, and Stucchi (2016) find that firms with strong linkages may participate in networks leading to different positive outcomes: reduced transaction costs, increased efficiency, stronger origination and sharing of tacit knowledge, and stronger and more effective cooperative action (e.g., asset and input-sharing). All of these outcomes will induce gains in efficiency and competitiveness at the cluster level. The benefits of industry clusters have gained attention in the public policy arena thanks to the works of Porter (1990, 1998, 2000). Governments throughout the world increasingly support CDPs to take advantage of agglomeration economies in their countries to increase productivity (Crespi, Fernández-Arias and Stein, 2014). The justification for government intervention, and therefore the existence of CDPs, lies in the presence of coordination failures and the provision of public goods (Maffioli, Petrobelli, and Stucchi, 2016). Although the objective of CDPs is to strengthen linkages 2 Public goods have two properties that make them unsuitable for market provision. Nonrivalry, meaning that, once produced, public goods could be used without limit by all actors in the agglomeration. The second property is limited appropriability, meaning that control mechanisms are highly ineffective in excluding free riders. Typical public goods are primarily information, such as new legislation, sectoral regulations, generic technological knowledge applicable to the sector, branding, and others. Misalignment of incentives will make Coasian self-regulating solutions out of reach. Thus, coordination of collective action through public policy is the only way to deliver these goods. 4 and relationships between firms within a cluster, they are only a tool to reach the final goal: stimulating productivity as a way to increase competitiveness. The presence of spillovers is a well-studied phenomenon in the economics of agglomeration literature. Because of their intrinsic characteristics, however, how to measure spillovers and the general equilibrium effects of cluster policies remains an open question in the literature. Researchers do not usually have sufficiently rich firmlevel data to correctly estimate them. Huber (2012) finds very little empirical evidence of the mechanisms of local knowledge spillovers and cautions academics and policymakers against making assumptions about the existence of spillovers in clusters. To the best of our knowledge, relatively few studies have successfully analyzed indirect and/or total effects of cluster development programs. One such study is FigalGarone et al. (2015). Using firm-level data on Brazilian SMEs for the period 2002– 09 and combining fixed effects with re-weighting methods, they estimate both the direct and the indirect effects of a cluster development program in Brazil on three variables: level of employment, value of exports, and probability of exporting. To estimate the indirect effects, the authors classify as indirect beneficiaries those firms that did not participate in the program and that were located in a municipality where there were direct beneficiaries in the same industry. The authors found positive spillovers in export outcomes and a negative effect on employment in the first year after the program. The latter effect may be coming from labor mobility from indirect beneficiaries to direct ones. While FigalGarone et al. (2015) define indirect beneficiaries using geographic proximity criteria, Castillo et al. (2015) identify them by labor mobility. Indirect beneficiaries are firms that hired workers that were working in a direct beneficiary firm. The program evaluated is an innovation program called FONTAR, which was carried out in Argentina between 1998 and 2013. The paper measures spillovers by the degree of performance improvement of firms that hired skilled workers from the treated firms. To estimate this effect, the study uses a lagged dependent variable model to compare these indirectly affected firms with a group of firms that had a similar evolution on key variables before they hired skilled workers from the participant firms. The authors find that the indirectly affected firms experienced increases in employment, wages, the probability of exporting, and the value of exports. The authors conclude that increased productivity drives these effects. In Boneu et al. (2014), the authors estimate the spillover effects associated with a technological cluster located in the city of Cordoba, Argentina. While the direct beneficiaries are the small and medium-sized firms in the city of Cordoba that make up the technological cluster, the indirect beneficiaries are the same types of firms located 5 on the outskirts of the city. The authors used a panel of firms in the information and communications technology (ICT) sector for the period 2003–11, which allowed them to control for the dynamics of firm sales and fixed effects, applying a generalized method of moments estimator. The paper finds that for every new participant in the program, sales of nonparticipant firms increase by approximately 0.7 percent. Closely related to the last paper, Castillo et al. (2015) investigate the impact of a tourism policy on employment in the province of Salta, Argentina. Following the synthetic control method, they use a combination of untreated Argentinean provinces to construct a synthetic control province that shares relevant characteristics with Salta before policy implementation. They find that the CDP increased tourism employment in Salta by an average of 11 percent per year, for an overall impact of around 110 percent between 2003 and 2013. In this paper, we follow a similar approach to estimate the aggregate effect of a cluster policy in Uruguay. The paper adds to the scant literature on this subject by rigorously and quantitatively measuring the total (direct and indirect) effects of cluster policies. The purpose of the cluster policy that we focus on here was to increase the competitiveness of the tourism sector of a region in Uruguay (Colonia). The total investment in this policy between 2008 and 2014 was approximately $900,000. It attempted to develop business linkages, improve soft tourist infrastructure, and improve Colonia’s strategy to promote and market Colonia as a tourist destination. The city of Colonia has some particular characteristics that make it one of the most popular destinations for tourists visiting Uruguay. First, UNESCO declared it a World Heritage Site in 1995. Moreover, one-quarter of all tourists visiting Uruguay enter the country through Colonia's port. The city is only 50 km away from Buenos Aires, Argentina's capital. 3. The PACC Program 3.1. Program Description The PACC was created in 2005 with the support of the Inter-American Development Bank. Its aim was to contribute to the development and competitiveness of clusters and supply chains. Since its inception, the PACC has worked with 21 clusters. Each cluster intervention has three components: a strategic plan, matching grants, and strengthening of the supporting institutions of the cluster. The PACC had two main stages: (i) cluster selection and preparation of competitiveness strengthening plans, and (ii) execution of projects and actions to strengthen public and private supporting institutions (Figure 1). The process starts with 12 Source: Authors’ elaboration. 4. Data and Descriptive Statistics 4.1. Data The main data source used in this paper is the Survey on Receptive Tourism (Encuesta de Turismo Receptivo), which provides disaggregated information on Uruguay’s main seven tourist destinations: Colonia, Costa de Oro, Montevideo, Pirápolis, Punta del Este, Rocha, and the thermal littoral.3 For each of these regions, there is quarterly information on the number of visitors, tourists’ expenditures, and visitors’ average length of stay between 2000 and 2015. Map 1. Tourism Regions in Uruguay Colonia Litoral Montevideo Costa de Oro Piriápolis Punta del Este Rocha Source: Uruguay XXI Institute, Tourism and Real Estate Report (2011). We are also using information from the Continuous Household Survey (Encuesta Continua de Hogares, or ECH) conducted in Uruguay. Specifically, residents’ average household income was used as a proxy for the level of development of each region.4 3 The remaining destinations are grouped in a residual category. 4 Although a priori the ECH provides several variables that describe the residents and the labor market of each region, the nature of the sample makes it difficult to obtain precise medians at the region level. This problem gets worse when attempting to construct sector-level variables in each region, such as the number of people employed in the hotel and restaurant sector. In fact, there was no gain in the mean prediction error when synthetic controls are constructed, including predictors using information from the ECH. 14 4.2. Descriptive Statistics Between 2000 and 2002, the economic crisis that plagued the region, particularly the Argentinean crisis (see Figure 3), did not spare the tourism sector. The number of visitors recovered slightly between 2003 and 2004, remaining stagnant until 2007. Between 2007 and 2011, there was significant growth. Between 2011 and 2015, the number of tourists again remained flat, while their average spending fell starting in 2013. Figure 3. Number of Tourists and Annual Expenditure Blockade of the General San Martín Bridge Cluster tourism policy in Colonia Foreign exchange controls in Argentina 1 1.5 2 2.5 3 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 050 100 150 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 year Source: Authors’ elaboration based on information from the Ministry of Tourism. Note: For the seven tourism regions in Uruguay. International tourism demand to visit Uruguay is highly concentrated in the Colonia region, especially from Argentina (57 percent of the total) and Brazil (15 percent of the total). Hence, Argentina’s demand determinants significantly affect Uruguay’s tourism performance. This is especially true for Colonia, given its proximity to Buenos Aires. These determinants include macroeconomic variables, especially the evolution of the level of activity and the bilateral exchange rate, but also economic policy measures taken by Argentina that affect Uruguay’ s tourism sector. With respect to level of activity, Argentina’s GDP growth between 2003 and 2011 positively affected tourism demand. Between 2003 and 2008, average annual GDP growth in Argentina was 9 percent. 2009 saw a contraction in the level of activity 15 due to the international crisis, but it recovered quickly and the economy again reported high growth rates (8 percent on average) between 2010 and 2011. Between 2012 and 2015, the Argentinean economy was stagnant. The Uruguayan currency tended to appreciate with respect to the Argentinean peso in this period, with the exception of 2002 and 2003, when there was a sharp devaluation of Uruguayan peso (see Figure A.1 in the Appendix). With respect to Brazil, however, Uruguay gained in competitiveness until 2007, after which it fell behind, resulting in a bilateral exchange rate in 2015 near the one observed in 2000. Because of the close proximity of Colonia to Buenos Aires, it is important to consider some events in the period of analysis that may have affected the evolution of the inflow of Argentinean tourists and the level of their expenditures. The first was the dispute between the Uruguayan and Argentinean governments over the location of a pulp mill on the banks of the Uruguay River in the city of Fray Bentos. Between 2005 and 2010, Argentinean residents of Gualeguychú and green organizations mobilized against the mill’s construction. One of the most important actions taken was the blockading of the Libertador General San Martín Bridge, a border-crossing point between the two countries. Although the effect of the bridge closure on the influx of Argentinean tourists to Uruguay was clearly negative, it is less clear that it negatively affected tourism to Colonia. This is because, for the people of Buenos Aires, Colonia became an alternative entry/exit point to and from Uruguay. Figure A.3 in the Appendix shows the significant decrease in the influx of tourists through Fray Bentos and a simultaneous increase in entries through Colonia. However, the greater importance of Colonia as an entry/exit point does not imply that it became more important as a tourist destination—although it is not possible to discount the bridge blockade as a positive externality with respect to the influx of Argentinean visitors.5 This represents a constraint on identifying the impact of the cluster policy, since the bridge blockade went beyond 2007, the same year that the tourism cluster in Colonia began to develop. Nonetheless, Colonia’s largest revenue growth occurred in 2006 and 2007, prior to the start of the program. Figure A.5 in the Appendix shows the number of tourists as a proportion of the number of people who entered through the Port of Colonia. Until 2004, this proportion was approximately 0.4. It decreased significantly between 2006 and 2007 until it reached a floor of 0.26. This result is consistent with the fact that 5 A similar effect may be expected in the thermal littoral (which includes areas in the departments of Paysandu and Salto), as the other two land crossing points with Argentina are in this region. The latter was an alternative entry/exit point for residents of other provinces but not those in Buenos Aires. 16 Colonia became more important for Argentineans as an entry point to the country. However, the number of tourists in Colonia increased rapidly starting in 2008 despite the fact that the blockade of the Libertador General San Martín Bridge continued until 2010. Another important factor is Argentina’s foreign exchange policy. In November 2011, the Argentinean government imposed an exchange rate control to curb capital outflows. It imposed restrictions on the purchase of foreign currency for the purpose of foreign travel (known as the exchange-rate trap, or “cepo cambiario”). From then on, a plethora of other measures to prevent people from eluding exchange-rate controls were put in place. A popular practice among Argentineans consisted of traveling to Uruguay (mainly to Colonia) to withdraw cash in dollars from automatic teller machines at the official exchange rate (cheaper than the parallel market rate in Argentina). The Argentinean authorities imposed measures, such as a surcharge of up to 35 percent on purchases made using international credit or debit cards and maximum limits on money withdrawn abroad, to counterbalance these practices.6 The exchange rate restrictions (which lasted until the end of 2015), as well as the measures that sought to limit Argentineans’ spending in foreign countries, had an impact on tourism demand in Uruguay. As Figure 2 shows, 2011 marked the beginning of a standstill in tourist arrivals and expenditures. It is also important to highlight that the Argentinean economy entered a phase of stagnation at the same time. However, the effect of these episodes on international tourism demand in Colonia is, a priori, ambiguous. First, the policy represented an important constraint on Argentinean demand, which may have affected tourism in Colonia. Second, because Colonia became an attractive place for Argentineans to use credit cards to avoid the foreign exchange restrictions, a positive externality on tourism demand in Colonia cannot be discounted. The evolution of the number of tourists and their expenditures in the seven regions of the country starting in 2011 (see Figures A.2 and A.3 in the Appendix) shows that growth slowed in both Colonia and the Litoral (the two tourism regions bordering Argentina), but tourism and expenditures declined in the remaining regions. Since our main objective is to identify the effects of a cluster tourism policy that began in 2007, we can be relatively confident that between 2008 and 2010 this impact was not distorted by the episodes described above. Starting in 2011, the difference between what happened in Colonia and the control group may be distorted by these 6 A similar practice occurred with the Uruguayan casinos that used US dollars. Argentinians went to the casinos, spent a certain amount of money on chips, and then exchanged those chips for U.S. dollars at the official exchange rate. 17 events. However, the direction of the bias is not clear, given the likelihood that the Litoral region, which is in the control group, was affected in similar way as Colonia. Figure 4 shows the number of international tourists visiting Colonia and all of the remaining tourism regions between 2000 and 2016. 2008 was a turning point, and it is also the year that the cluster program started. If we observe each of the other regions individually (see Figure A.2 in the Appendix), it is also possible to observe a shift in the trend that year in the Litoral region and on the coast of Rocha. If we compare 2009 to 2006, the total number of international tourists visiting Colonia increased by 5.4 percentage points (see Figure 3).7 In terms of total expenditure (Figure 5), the differences in trends between Colonia and the other tourism regions since the start of the cluster program are not clear. Although tourism expenditure increased at a higher rate in Colonia between 2009 and 2010, it also contracted more steeply starting in 2014. Colonia’s participation in total expenditure between 2006 and 2009 increased by 2.2 percentage points (from 3.2 percent to 5.4 percent) (Figure 6). Figure 4. Number of International Tourists by Tourism Regions of Uruguay (seasonally adjusted) 0 200 400 600 800 1000 0 10 20 30 40 50 60 70 80 2000q3 2001q2 2002q1 2002q4 2003q3 2004q2 2005q1 2005q4 2006q3 2007q2 2008q1 2008q4 2009q3 2010q2 2011q1 2011q4 2012q3 2013q2 2014q1 2014q4 2015q3 2016q2 Tourists: Colonia (thousands) Tourists: All other regions (right axis) Source: Authors’ elaboration based on information from the Ministry of Tourism. 7 We consider the year 2006 as the last pre-program year, since the plan for the Tourism Colonia cluster was approved in 2007 and the first structural project was approved in 2008. 18 Figure 5. Regional Participation in International Tourism to Uruguay, 2006 and 2009 (in percent of expenditure) 35.4 27.8 11.4 11.4 4.2 3.9 5.9 41.6 31.2 11.4 6.0 3.3 3.0 3.5 Montevideo Punta del Este Litoral Colonia Costa de Oro Pirápolis Rocha 2006 2009 Source: Authors’ elaboration based on information from the Ministry of Tourism. Figure 6. Total International Tourism Expenditure by Region (seasonally adjusted) 0 100 200 300 400 500 600 700 0 10 20 30 2000q3 2001q2 2002q1 2002q4 2003q3 2004q2 2005q1 2005q4 2006q3 2007q2 2008q1 2008q4 2009q3 2010q2 2011q1 2011q4 2012q3 2013q2 2014q1 2014q4 2015q3 2016q2 Spending: Colonia (millions of USD) Spending: All other regions (right axis) Source: Authors’ elaboration based on information from the Ministry of Tourism. 19 Figure 7. Regional Participation in International Tourism to Uruguay, 2006 and 2009 (in percent of total) 33.4 46.5 4.0 5.4 1.9 3.9 5.0 36.9 48.0 4.3 3.2 2.2 2.6 2.8 Montevideo Punta del Este Litoral Colonia Costa de Oro Pirápolis Rocha 2006 2009 Source: Authors’ elaboration based on information from the Ministry of Tourism. 5. Empirical Strategy To analyze the impact of PACC on the tourism sector in Colonia, this study focuses on demand indicators. It is based on information provided by the Receptive Tourism Survey, specifically the number of international tourists and their expenditures. Since the treatment unit is a region and at the same time there is a group of aggregated units (or regions) that could serve as a potential control group, the synthetic control method appears to be an appropriate technique. Moreover, given the difficulty of applying traditional impact evaluation methods in this case, applying this method is e the only way to produce a rigorous quantitative impact evaluation. Synthetic control methods have been used to study economic impacts caused by several different events. They were used, for example, to measure the effects of terrorists attacks (Abadie and Gardeázabal, 2003), natural disasters (Cavallo et al., 2013), particular economic regimes (García Ribeiro, Stein, and Kang, 2013), tobacco control policies (Abadie, Diamond, and Hainmueller, 2010), major sporting events (García Ribeiro et al., 2015), and tourism development policies (Castillo et al., 2015). The synthetic control method assigns a weight to each unit in the control group according to an optimization process which minimizes the distance between vectors that contain information related to the variables of interest for the period before the intervention, for the treated and control units. Following Abadie, Diamond, and Hainmueller (2010), we define as the indicator of treatment for region j at moment t. 20 The observed outcome variable equals the sum of the effect of the treatment () and the counterfactual which is specified as a factor model: (1) where is a unknown common time effect, is a vector (rx1) of observed covariates not affected by the treatment, is a vector(1xr) of unknown parameters, is a vector (1xF) of observed common factors, is a vector (Fx1) of unknown factorial loads, and is a zero mean independent error. If j=1 is the region affected by the policy, the treatment effect is estimated by approximating the unknown with a weighted average of untreated regions. We call the number of periods before the treatment, the total periods and j the observed regions where the first is the unit which receives treatment and the rest are “donors.” is the results vector (Tx1) for unit j and is the results matrix (TxJ) of all donors. W is a weights vector (Jx1)of all donors observations, , so and . The weighted mean of donors is constructed as . is the partition between pretreatment and posttreatment results vectors. represent the combination of k predictors, which include the r covariates y M linear combinations of (k=r+M). Analogous, is the matrix (kxJ) of predictors for donors. The synthetic control method consists of finding the optimal weighting matrices in such a way that the difference of the predictors’ values of the treated and the counterfactual becomes as small as possible: (2) In this way, the treated region and its synthetic control are similar along the dimensions that matter to predict the outcome variable prior to the treatment. is a nonnegative diagonal matrix (kxk) whose values represent the weights of the predictors, that is, the values that prioritize which predictor matches better in (2).8 The inference process is valid for any set of predictor weights, but Abadie, Diamond, and Hainmueller (2010) suggest choosing the set of weights that minimize the root mean squared prediction error (RMSPE) in the pretreatment period. 8 As matching may hold only approximately. 21 Under specific conditions, Abadie, Diamond, and Hainmueller (2010) show that the bias of tends toward zero when the number of periods before the treatment ( ) increases in relation to the scale of . The synthetic control obtained is a good approach to the counterfactual, and thus, its path posttreatment reflects what would have happened with the treated region in the absence of the intervention. To determine statistical significance, placebo tests are performed. These consist of taking each region from the control group and applying the same method as if it were a treated unit (excluding the treated region from the respective synthetic control) to obtain a distribution of the placebo effects. If the distribution contains effects as large as the effect of a truly treated unit, then we should assign a high probability that the effect has occurred by chance. This non-parametric test has the advantage of not imposing any error distribution. Formally, if is the distribution of the placebo effects, then the p-value of the estimated effect is the following: The p-value is interpreted as the proportion of the control group units which have an estimated effect at least as large (in absolute value) as the treated unit. It is important to note that inferences using these p-values can be overly conservative given that placebo effects may be large in cases where a good adjustment in the pretreatment period is not achieved for the placebo regions (i.e., a good synthetic control is not achieved). An alternative is to divide the effects by RMSPE in the pretreatment period ( ), and obtain a pseudo t-statistic for each posttreatment period, . Similarly, both statistics can be defined for the entire post-intervention interval using the RMSPE in that interval ( ). Therefore, the p-value for the joint significance of the effects in all posttreatment periods is defined as the proportion of placebos which have at least as large as the treated unit: 28 Table 4. Robustness of Impact Excluding Regions from Donor Group Source: Authors’ elaboration based on information from the Ministry of Tourism. A second robustness analysis is related to the date established to determine the preand post-program period for the empirical exercises. Up to this point, the first year after the intervention was 2008. This will now be changed. The exercise consists of replicating the same analysis but assuming alternative starting dates: 2007, 2006, 2005, and 2004. This exercise is interesting to analyze whether events that occurred in those years may be the real cause of the estimated effect after 2008. For example, it could be useful to discover whether the blockade of the bridge, which began in 2005 and escalated in 2006, could be the real event behind the positive effect found after 2008, or at least one of its causes. The results (Figure 11) show that even when a different starting date is used, it is not until 2008 that a positive gap for Colonia was observed. In other words, the previous estimated effect does not change significantly if we estimate and “release” the synthetic control well before 2008. Figure 11. Impact on the Number of Tourists, Assuming Different Starting Dates of the Intervention 20 40 60 80 2000q1 2005q1 2010q1 2015q1 quarter Colonia Synth_2008 Synth_2007 Synth_2006 Synth_2005 Synth_2004 synthetic Colonia: placebo starting date (1/2/3/4 year before) -10 010 20 30 2000q1 2005q1 2010q1 2015q1 quarter estimated effect: placebo starting date (1/2/3/4 year before) Source: Authors’ estimations based on information from the Ministry of Tourism. Excluding from donors Average effect (thousands of tourists) Costa de Oro 12.7 0.5 Litoral 21.4 0.0 Montevideo 15.5 0.0 Costa de Oro, Litoral 19.9 0.0 Costa de Oro, Montevideo 6.8 0.5 Litoral, Montevideo 20.7 0.0 Costa de Oro, Litoral, Montevideo 18.6 0.0 29 A final aspect to be considered is related to the decision regarding the selection of predictors. First, it should be noted that we only have two covariates to be used as additional predictors of lagged values of outcome. Kaul et al. (2016) demonstrate that using all outcome lags as separate predictors renders all other covariates irrelevant. This holds regardless of how important these covariates are in accurately predicting posttreatment values of the outcome, threatening the estimator’s unbiasedness. We will show the sensitivity of the results to alternative restrictions on the outcome variable lags. Following Kaul et al. (2016), we analyze two alternatives for the outcome variable: (i) including only the pre-intervention average of the outcome variable; and (ii) using only the last pretreatment value of the outcome variable. Previously, in Table 2, we observed the average adjustment achieved between Colonia and the synthetic control. Table 5 shows how this adjustment improves if we consider alternative versions of inclusion of the outcome variable as a predictor. Naturally, the average of regions that make up the synthetic control is different in each case, although in all cases the preponderance of Costa de Oro and Litoral is maintained (see footnote in Table 5). Figure 12 presents the estimated effect according to the three previous alternatives for the outcome variable. The impact is positive in all three cases, although it is not significant in the option in which only the first lag of the outcome variable is considered as a predictor. Table 5. Covariates (Predictors) Means before Treatment under Alternative Forms of Inclusion of Outcome Variable as a Predictor: Colonia vs. Synthetic Colonia Colonia Synthetic Colonia Yearly average of outcome variable(all lags included) (a) Average of outcome variable (pretreatment period) (b) Average of outcome variable in 2007 (c) Spending per tourist (USD) 2001q1-2007q4 160.3 163.8 160.4 160.3 Total spending (millions of USD) 2001q1-2007q4 4.2 4.8 4.2 4.2 RMSPE ( 2.5 2.6 3.2 Source: Authors’ estimations based on information from the Ministry of Tourism. Note: The respective synthetic controls are formed by the following regions (weights): (a) Montevideo (0.03), Costa de Oro (0.81) and Litoral (0.16); (b) Costa de Oro (0.72), Piriápolis (0.02), Rocha (0.02) and Litoral (0.24); (c) Costa de Oro (0.58), Rocha (0.12), and Litoral (0.30). 30 Figure 12. Estimated Impact on Number of Tourists under Alternative Predictors Significant -10 010 20 30 2000q1 2005q1 2010q1 2015q1 quarter Yearly average of outcome variable (all lags) Average of outcomes variable (pre-treat. period) Average of outcome variable in 2007 Source: Authors’ estimations based on information from the Ministry of Tourism. Finally, we cannot rule out the possibility of spillover effects on other regions. For example, marketing Colonia as a tourist destination abroad could have a positive impact on tourism in other regions; therefore, the impact that we are estimating could be downward biased. There could also be a stealing effect of tourists from other regions. Even though we cannot rule out this last hypothesis, one reason to think that this effect could be small is that tourism in Colonia is cultural heritage tourism that is different from tourism in the other regions, with positive weights in the synthetic control (more related to sun and beach tourism). 6.2. Total Expenditure The same exercise described in the previous section was performed with the variable “total expenditure” by tourists in Uruguay. In this case, we do not find a significant difference between Colonia and the synthetic control (Figure 13). The gap between them oscillates around 0 (Figure 14). The differences in each year, sometimes positive and sometimes negative, is never significant considering the placebo tests (see righthand panel, Figure 15). As a result, the impact in the entire posttreatment period is not significant (Table 6). Therefore, although a positive impact on the inflow of tourists to Colonia was identified, there is no evidence of an increase in their total expenditure. This conclusion remains unchanged if the period of time used to identify the synthetic control is modified (see Figure 15). 31 The positive impact previously estimated on the number of tourists and the null impact on total expenditure imply that average expenditure per tourist decreased in the posttreatment period. In the Appendix, the results obtained from repeating the same exercises as above for the variables “average expenditure per tourist,” “average length of stay,” and “average daily expenditure per tourist: are presented (Figures A.6–A.8). A negative gap is observed for the three variables, even though the results were not statistically significant in any of the cases. A possible hypothesis to explain the decrease in the average expenditure per tourist is that the demand induced by the program may have captured a different type of tourist that spends less (i.e., from different socioeconomic strata, having different habits, etc.). Figure 12. Expenditure: Colonia vs. Synthetic Control, 2000:01–2016:03 010 20 30 2000q1 2005q1 2010q1 2015q1 quarter Treated Synthetic Control Source: Authors’ estimations based on information from the Ministry of Tourism. 32 Figure 13. Expenditure: Estimated Impact -6 -3 0 3 6 2000q1 2005q1 2010q1 2015q1 quarter Source: Authors’ estimations based on information from the Ministry of Tourism. Figure 14. Estimated Impact on Expenditure: Colonia vs. Placebos (left) and Significance Tests (right) -20 -10 010 20 2000q1 2005q1 2010q1 2015q1 quarter Colonia Costa de Oro Piriápolos Rocha Litoral Source: Authors’ estimations based on information from the Ministry of Tourism. 33 Figure 15. Impact on Expenditure Assuming Different Starting Dates for the Intervention 010 20 30 2000q1 2005q1 2010q1 2015q1 quarter Colonia Synth_2008 Synth_2007 Synth_2006 Synth_2005 Synth_2004 synthetic Colonia: placebo starting date (1/2/3/4 year before) -5 0 5 2000q1 2005q1 2010q1 2015q1 quarter estimated effect: placebo starting date (1/2/3/4 year before) Source: Authors’ estimations based on information from the Ministry of Tourism. Table 6. Root Mean Square Prediction Error of Total Expenditure: Colonia vs. Placebos Region Quality of the pretreatment matches: Joint effect across all posttreatment periods: Adjusted effect (Post/Pre RMSPE) Colonia 0.7 1.9 2.7 Punta del Este 24.4 67.6 2.8 Montevideo 8.8 15.6 1.8 Costa de Oro 0.7 10.4 15.0 Piriápolis 0.8 2.7 3.5 Rocha 1.0 9.0 8.8 Litoral 3.3 7.0 2.1 p-values: 1 0.66 Source: Authors’ estimations based on information from the Ministry of Tourism. 7. Conclusions This paper analyzes the impact of a cluster tourism policy in the region of Colonia, Uruguay. A comparative analysis between Colonia and other tourism regions of the country was performed applying a synthetic control method. This method of identifying the counterfactual is especially useful in comparative case studies where there are a limited number of control units. The synthetic control method assigns a weight to every unit of the donor group according to an optimization process that minimizes the distance between vectors that have information related to the interest variables for the period before the intervention, for the treated unit and the controls. 34 The estimations show a positive impact of the cluster program on the inflow of international tourists to Colonia. The estimated impact was 14,000 tourists per quarter between 2008 and 2015, a 24 percent increase in the number of tourists in the period. Regarding the significance of the impact in the years following the start of the program, the evidence shows that the impact was always significant except for 2011 and 2012, when the difference between Colonia and its counterfactual is not significant. This may be attributable to the capital controls imposed by the Argentinean government on Argentinean tourists. We did not find a significant impact on total expenditure by tourists. This could be explained by a composition effect in the total number of tourists arriving to Colonia. We have different hypotheses for this. First, the incremental number of tourists could have been concentrated in segments of lower relative income. Second, the program may have attracted tourists who were less interested in the attractions of Colonia than those who had visited Colonia in the past but who, swayed by the marketing campaigns, decided to visit Colonia for only few days. Alternatively, border mobility and foreign exchange restrictions in Argentina may have adversely affected the quality of tourism to Colonia (i.e., length of stay in Colonia was less and/or spending was lower). Given these results, it is interesting to ask whether the program benefited the private sector. Without raising income, the program could have had a positive impact on firms’ profits if it reduced costs. Unfortunately, this study did not yield enough data to know if this is indeed the case. However, some program activities, such as worker training and management improvement, may have had an impact on costs. 35 References Abadie, A., A. Diamond, and J. Hainmueller. 2010. “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program.” Journal of the American Statistical Association 105(490): 493–505, June. Abadie, A. and J. Gardeázabal. 2003. “The Economic Costs of Conflict: A Case Study of the Basque Country.” American Economic Review 93(1): 113–32. March. Arrow, K. 1962. “Economic Welfare and the Allocation of Resources for Invention,” in R. Nelson (Ed.). The Rate and Direction of Inventive Activity, pp. 609–25. Princeton, NJ: Princeton University Press. Boneu, F., D. Giuliodori, A. Maffioli, A. Rodríguez, and R. Stucchi. 2014. “The Spillover Effects of ICT Cluster Support in Córdoba.” Available at: https://mpra.ub.unimuenchen.de/60307/1/MPRA_paper_60307.pdf Castillo V., L. FigalGarone, A. Maffioli, and L. Salazar. 2015. “Tourism Policy, a Big Push to Employment: Evidence from a Multiple Synthetic Control Approach.” IDB Working Paper No. IDB-WP-572. Washington, D.C.: InterAmerican Development Bank. Castillo, V., L. Figal Garone, A. Maffioli, S. Rojo, and R. Stucchi. 2015. “Knowledge Diffusion Through Workers Mobility.” Washington, D.C.: Inter-American Development Bank. Unpublished. Cavallo, E., S. Galiani, I. Noy, and J. Pantano. 2013. “Catastrophic Natural Disasters and Economic Growth.” Review of Economics and Statistics 95(5): 1549–61, December. Crespi, G., E. Fernández-Arias, and E. Stein. (Eds.). 2014. Rethinking Productive Development: Sound Policies and Institutions for Economic Transformation. Development in the Americas (DIA) 2014. Washington, D.C.: Palgrave Macmillan and Inter-American Development Bank. Delgado, M., M. Porter, and S. Stern. 2014. “Clusters, Convergence, and Economic Performance.” Research Policy 43(10): 1785–99. Ellison, G. and E. Glaeser. 1997. “Geographic Concentration in U.S. Manufacturing Industries: A Dartboard Approach.” The Journal of Political Economy 105(5): 889–927. FigalGarone, L. et al. 2015. “Cluster Development Policy, SME Performance, and Spillovers: Evidence from Brazil.” Small Business Economics 44: 925. doi:10.1007/s11187-014-9620-2. 36 García Ribeiro, F., G. Stein, and T. Kang. 2013. “The Cuban Experiment: Measuring the Role of the 1959 Revolution on Economic Performance using Synthetic Control.” Working Paper. Available at: http://economics.ca/2013/papers/ SG0030-1.pdf García Ribeiro, F., G. Pereira, J. de Araújo, and V. Halmenschlager. 2015. “Evaluación del impacto de la copa del mundo: Las copas de Francia (1998), Japón/Corea del Sur (2002) y Alemania (2006).” Estudios y Perspectivas en Turismo 24: 628–44. Glaeser, E. L., H. D. Kallal, J. A. Scheinkman, and A. Shleifer. 1992. “Growth of Cities.” Journal of Political Economy 100: 1126–1152. Huber, F. 2012. “Do Clusters Really Matter for Innovation Practices in Information Technology? Questioning the Significance of Technological Knowledge Spillovers.” Journal of Economic Geography 12(1): 107–126. IDB (Inter-American Development Bank) 2014. Rethinking Productive Development: Sound Policies and Institutions for Economic Transformation. Washington, D.C.: Inter-American Development Bank. Jaffe, A., M. Trajtenberg, and R. Henderson. 1993. "Geographic Localization of Knowledge Spillovers as Evidenced by Patent Citations." Quarterly Journal of Economics 63(3): 577–98. Kaul, A., S. Klößner, G. Pfeifer, and M. Schieler. 2016. “Synthetic Control Methods: Never Use All Pre-Intervention Outcomes as Economic Predictors.” Unpublished paper. Kerr, W. R. and S. D. Kominers. 2015. “Agglomerative Forces and Cluster Shapes.” The Review of Economics and Statistics 97(4): 877–99. Maffioli, A., C. Petrobelli, and R. Stucchi. 2016. "The Impact Evaluation of Cluster Development Programs: Methods and Practices." Washington, D.C.: InterAmerican Development Bank. Marshall, A. 1920. Principles of Economics. 8th Edition. London, United Kingdom: Macmillan. Porter, M. 1990. Competitive Advantage of Nations. New York, NY: Free Press. --------. 1998. “Clusters and the New Economics of Competition.” Harvard Business Review 76(6): 77–91. --------. 2000. “Location, Competition, and Economic Development: Local Clusters in a Global Economy.” Economic Development Quarterly 14(1): 15–34. Rius and Freira. 2016. "Clústers y conglomerados: el caso del turismo en Colonia." Unpublished. 37 Romer, P. 1986. “Increasing Returns and Long-run Growth.” The Journal of Political Economy 94: 1002–37. Rosenstein-Rodan, P. N. 1943. “Problems of Industrialization of Eastern and SouthEastern Europe.” The Economic Journal 53(210/211): 202–11.