Exporting democratic practices: Evidence from a village governance intervention in Eastern Congo

Humphreys, Macartan; de la Sierra, Raúl Sánchez; der Windt, Peter Van Article — Accepted Manuscript (Postprint) Exporting democratic practices: Evidence from a village governance intervention in Eastern Congo Journal of Development Economics Provided in Cooperation with: WZB Berlin Social Science Center Suggested Citation: Humphreys, Macartan; de la Sierra, Raúl Sánchez; der Windt, Peter Van (2019) : Exporting democratic practices: Evidence from a village governance intervention in Eastern Congo, Journal of Development Economics, ISSN 0304-3878, Elsevier, Amsterdam, Vol. 140, pp. 279-301, https://doi.org/10.1016/j.jdeveco.2019.03.011 This Version is available at: https://hdl.handle.net/10419/214632 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc-nd/4.0/ Exporting Democratic Practices: Evidence from a Village Governance Intervention in Eastern Congo Macartan Humphreys∗ Raul Sanchez de la Sierra† Peter Van der Windt‡ June 15, 2019 Abstract We study a randomized Community Driven Reconstruction (CDR) intervention that provided two years of exposure to democratic practices in 1,250 villages in eastern Congo. To assess impacts, we examine behavior in a later village-level unconditional cash transfer project that distributed $1,000 to 457 treatment and control villages. The exercise provides opportunities to assess whether public funds get captured, what governance practices are employed by villagers and village elites and whether the intervention altered these behaviors. We find no evidence for such effects. The results cast doubt on current attempts to export democratic practices to local communities. JEL Codes: D72; P48; D02; O17 Keywords: Political Processes; Political Economy; Institutions; Culture; Demonstration Effects Forthcoming in Journal of Development Economics. ∗Columbia University and WZB Berlin Social Science Center. [email protected]. †Corresponding author. UC Berkeley. [email protected]. ‡New York University - Abu Dhabi. [email protected]. This research was funded by the International Initiative for Impact Evaluation (3IE) and the Department For International Development, UK. We thank the International Rescue Committee, CARE International, and Chimanuka Bantuzeko for their partnership in that research. We thank Lily Medina and Clara Bicalho for excellent support with analysis. Humphreys thanks the Trudeau Foundation for support while this work was undertaken. This research was approved by Columbia University IRB and was implemented with consent of participating subjects. The study did not make use of deception at any stage. Van der Windt thanks Wageningen University. Hypotheses, econometric specifications and covariates were specified prior to the development of social science registries. See Humphreys et al. (2013); deviations from this plan are discussed in appendix L. 1 Introduction Research in political economy and long-run development suggests that institutions are a key driver of economic development. Since economic and political institutions constrain incentives, they can produce variation in the extent to which allocations benefit populations or elites, and, ultimately, economic growth (Acemoglu et al., 2001). Although political institutions are likely to change slowly, international aid organizations and Western governments have taken a cue that they are important and have sought to change them in developing areas where they deem them weak. One common approach is to provide short-term exposure to the practice of institutions that are inclusive, or more democratic, in the hope that these will get picked up and lead to greater accountability of local elites. A prominent version of this approach in post-conflict settings uses the “Community Driven Reconstruction” (CDR) model. CDR programs, a variant of Community Driven Development (CDD) programs, are part of the “participatory development” model for development – a popular model that accounted for $85bn in World Bank spending in one decade alone (Mansuri and Rao, 2013). While many participatory development programs seek to use participation to enhance program effectiveness or program legitimacy, many CDR programs stand out for their transformative ambitions, seeking not just to involve community institutions, but to refashion them. However, the belief that this approach can improve institutions hinges on two questionable assumptions. First, that governance practices of conflict-affected areas need to be changed. And, second, that exposure to foreign governance practices — believed to be better for the population — will be adopted. While some studies suggest that CDR programs have an effect on subsequent public goods games for subgroups (Fearon et al., 2009), there is almost no evidence that CDR programs have any subsequent effect on governance practices (Casey et al., 2013; Beath et al., 2013; King and Samii, 2014). In this paper, we study an unusually large randomized CDR program and ask whether it caused subsequent village allocations of public funds to be less captured by village elites and whether it induced more democratic governance practices. The program targeted 1,250 villages in the Democratic Republic of the Congo (henceforth, the Congo) to undergo democratic training and practice in the management of development funds over two years. It was implemented by two US non-governmental organizations (NGOs) operating in the Congo and had a total cost of $46m and a target beneficiary population of 1,780,000 people. During the program, villages were trained in the implementation of elections, in accounting, and in accountability practices and were exposed to advocacy for democratic processes (through awareness raising campaigns where “good governance” practices were introduced). Participating populations then put these ideas into practice by selecting a development project for 1 the village and electing a management committee that managed project funds. Populations were tasked with holding the committee accountable through frequent town hall meetings. This intervention thus created exposure to the democratic process at the village level, which donors believed to be sufficiently prolonged to induce a change in local governance practices. We explore the impact of the CDR program along two dimensions. We first focus on the allocation of public funds, particularly the capture of benefits by the elite. We then focus on three dimensions that measure the degree to which governance practices are democratic: inclusiveness of the process (participation), accountability of the elite (accountability), and transparency of the elite actions (transparency). To measure the allocation of public funds and the degree to which governance practices are democratic, we observe community behavior in a real public funds allocation decision after about two years of CDR program exposure. To induce a comparable public funds problem in each village, we provide village-level unconditional cash transfers of $1,000 to 457 villages, half of which had been previously treated by the CDR program, the other half not. We use this cash transfer project (dubbed “RAPID”) to observe how the community solves the public funds allocation problem, and the governance practices that it sets in motion. The RAPID cash transfer project was implemented in four steps and took two to three months in each community. Communities were free to use the funds as they chose and to decide how to manage the use of funds. The project was designed so that we could measure changes in the allocation of public funds and governance practices, using direct observations, surveys conducted in private at the start and end of the project to a random sample of households and to the chief, focus groups with villagers and the elite, and a comprehensive audit. We focus on transparency of information held by elites, participation of villagers, composition of the committee and kinship relations, funds misuse and corruption, predominance of villagers and chief’s preferences in influencing the public funds allocation in the RAPID cash transfer project. Our first result is that, despite the scale and duration of the CDR intervention, we find no evidence of impact on the extent to which there is capture of benefits by the elite, as measured by the allocation of benefits from the RAPID cash transfer program. Our second result is that the failure of the CDR program to affect elite capture is accompanied by a failure to affect democratic practices along the three dimensions that we measure. In particular, the program leaves unchanged the patterns of inclusiveness and participation in the governance processes in the village, the degree to which the population holds the elite accountable, and the level of transparency regarding the usage of public funds in the village. 2 Our third result is that these findings cannot be easily explained by low statistical power, poor CDR program implementation, spillovers to control areas, social desirability in control areas, poor measurement, elite backlash against democratization, or changes in expectations about future aid in control villages. Our findings are thus a strong indication that adoption of democratic institutions does not follow from two years of training and practice. Eastern Congo is a well-suited environment to examine the adoption of democratic practice in local governance. The state has largely withdrawn from the rural areas of the east and enjoys low legitimacy. Local governance is often described as “captured” by traditional chiefs and vulnerable to corrupt practices by state officials. These features are not unique to the Congo. Multiple accounts suggest that in many Sub-Saharan states, colonial rule used pre-colonial institutions to create “decentralized despots” in ways that are detrimental to development (Acemoglu et al., 2014a; Mamdani, 1996). The types of democratic practices introduced by this program are believed to be largely new in the rural areas, where chiefs inherit power through lines of succession and where chief accountability hinges on other outcomes than is usually assumed by Western donors, practitioners, and academics (Hoffmann, 2014). This study complements political economy and development literatures that study the role of institutions. The view that liberal democratic institutions, inspired by Western models, are more conducive to growth finds support in academic research (Acemoglu et al., 2001; Acemoglu and Robinson, 2012). Furthermore, our study has implications for development policy, at a moment where billions are being spent by Western donors and international organizations in transforming institutions with little evidence to support the methods or the objectives of such an approach (Mansuri and Rao, 2013). In addition, our study makes three methodological contributions. First, we develop a novel measure of governance practices that hinges on observation of behavior of typical households and elites in order to obtain a comprehensive characterization of the social process that underpins them. By introducing real economic trade-offs, this paper’s approach to measurement improves upon standard self-reported measurements, which are subject to reporting and desirability bias (Barron et al., 2009). It also improves upon laboratory games, where interpretation of effects can be rendered difficult by the absence of natural metrics or sensitivity to features of artificial environments (Fearon et al., 2009; Lowes et al., 2017; Haley and Fessler, 2005; Avdeenko and Gilligan, 2015). Although our strategy is similar to some of the activities studied by Casey et al. (2013) and Beath et al. (2013), our measurement strategy complements these studies in that the behaviors that we observe occur in a large-stakes environment, in a forum with minimal control by the researchers, and 3 over an extended period of time. It is thus more straightforward to interpret our results as “natural” governance processes and outcomes of the community. While the large number of outcomes introduces multiple comparisons problems, which the analysis addresses, it also allows us to characterize governance practices more comprehensively than previous studies. Second, we address the concern that the absence of effects identified in this and other studies could arise because of positive spillovers to control areas. This is a particularly important concern, because the treatment is basically information and practice, which can easily spread through social networks, especially if beneficiaries think it is a positive adoption. Indeed the underlying theory behind these interventions assumes that exposure induces takeup. However, to date, no study of CDR or CDD has tackled the issue of spillovers. Using a model of indirect effects, our approach allows us to examine spatial spillovers in a way that makes use of exposure propensities implied by the randomization procedure and that hinges on relatively few assumptions (see Aronow and Samii (2013)). Third, to our knowledge, our study provides the results of the largest CDR program experiment to date. The CDR program was implemented in 1,250 villages across 560 village clusters. Our measurement strategy employs data collected in 457 village clusters, significantly reducing the likelihood of false negatives. As a comparison, Fearon et al. (2009) examine 83 villages, Casey et al. (2013) examine 236 villages, and Beath et al. (2011) examine 217 village clusters (Casey, 2018). Section 2 discusses the intervention we study in the context of the literature and related theory. Section 3 anchors the study in the Congolese context. Section 4 presents the details of the intervention. We present the empirical strategy and the results in Sections 5 and 6. Section 7 discusses whether the null effect is due to weaknesses of the experimental design. We conclude in Section 8. 2 Research Question and Existing Literature We examine an ambitious implementation of the CDR model. The CDR program that we examine, supported by the UK Department For International Development (DFID) with a budget of $46 million, was one of the largest programs of its form. This led the CDR program to be called “one of the world’s largest ever randomized trials” (see Hartford (2014) and Hartford (2012)). Donors were enthusiastic about the transformative effects of this CDR program on governance culture in the villages targeted. From data we gathered before the start of data collection, most decision-makers at the implementing agency thought it possible or very likely that the CDR program would have an impact on each dimension of 4 governance. Half of them thought it very likely that CDR program villages would become more transparent and inclusive.1 Other studies also examine whether participatory development can alter governance practices (see especially reviews by King and Samii (2014) and Casey (2018)). We differ from these studies in the following ways. First, while Fearon et al. (2009) examine whether bringing people together increases the valuation of the public good, our study focuses instead on whether democratic governance practice can be adopted, leading to a change in political institutions. Second, while Beath et al. (2013) examine whether state aid increases state legitimacy by creating reciprocity, we focus instead on external aid as the vehicle of transmission of governance practices. This distinction is also policy relevant, since aid is often a promising lever of change when the state is ineffective. Our study is closest to Casey et al. (2013), but complements it in a number of ways, besides providing variation in location and scale. While Casey et al. (2013) examine a program that “reconstitutes elected district-level governments,” we examine a CDR program that introduces exposure to parallel, and new, democratic institutions and aims to induce change in governance practice rather than reinforce the capacity of recently created institutions of governance. The intervention in Casey et al. (2013) aims to promote the effectiveness of institutions that are part of an administrative system that already exists — created just after the war — and that will continue to exist after the program, trying to foster participation and inclusion. The phase of the program we study also reflected the desire to focus on institution-building rather than infrastructural development. As we describe below, only $16m out of the $46m budget in the CDR program we study went towards the actual infrastructure projects, compared to an approximate one-to-one allocation in Casey et al. (2013), and the facilitation costs were front-loaded for the first stage of the program. Finally, there are important differences in measurement strategy. Casey et al. (2013) use innovative structured exercises to observe behavior, such as decisions over the choice of a battery or salt in the presence of enumerators or the use to which a gift of tarp was put. In contrast, our strategy sought to emulate a realistic and unstructured project management problem with minimal control over what choices should be made or what institutions should be used. In the appendix (A), we also discuss how this study relates to more theoretical literature, illustrating the mechanisms through which exposure to democratic practices might alter political institutions. 1Prior to launching our endline data collection, we conducted a survey with CDR program implementers and program directors (12 respondents) and seven researchers working in the region. Researchers, in contrast, were considerably more skeptical that traditional leaders would become more accountable. 5 3 Governance Practices in Eastern Congo The donors that supported the CDR intervention aimed to change local governance practices in eastern Congo. We provide here a short description of pre-existing governance practices to clarify that the intervention we examine took place in a setting where institutions of governance already existed and were entrenched, rather than in one in which institutions were destroyed and needed to be reconstructed. Public authority in rural areas of eastern Congo is mostly embedded in customary chiefs (Newbury, 1991), which are positions of informal power that predate the colonial state. In addition, in the North and South Kivu provinces particularly, non-state armed organizations collect regular taxes, provide protection, and run fiscal administrations (Hoffmann et al., 2016; Stearns et al., 2013; Stearns and Vogel, 2015; Raeymaekers, 2014; Sanchez de la Sierra, 2017). Customary chiefs can be one of two types: a village chief or a chiefdom chief, the “Mwamis.” Mwamis control a larger area, called “chefferie,” which is composed of tens to hundreds of villages. Village chiefs derive their power, in principle, from a set of governance practices that have been in place for generations (Akyeampong et al., 2014). By coordinating expectations, these practices are embedded with certain forms of legitimacy (Hoffmann, 2014; Newbury, 1991). Village chiefs are often enthroned following kin based lines of succession. When a village chief is enthroned, the chiefdom chief, with the help of local witch doctors, invokes the tribal ancestors to confirm the legitimacy of the new chief. Once enthroned, a village chief usually governs for life. The colonial state re-inforced the power of village as well as chiefdom chiefs. Indeed, the Belgian administrators co-opted customary chiefs, and obtained taxes, labor, and other resources through them, in exchange for the support of the coercive apparatus of the colonial state (Hoffmann, 2014; Acemoglu et al., 2014b; Mamdani, 1996). After independence, village and chiefdom chiefs remained as a basis for public authority. Village chiefs are the owners of land which, according to custom, is where ancestors are buried, and which they can allocate to households in exchange for a tax. Village chiefs most often administer justice and taxation in the village. They also organize the provision of public goods (clearing the road, building infrastructure, and mobilizing self-defense groups), drawing on an old tradition of forced labor, Salongo. The power of village chiefs can hinge on perceptions of their supernatural talent, their toughness, and their leadership skills (Newbury, 1991). 6 4 Intervention: Community Driven Reconstruction Our study takes advantage of a large UK funded CDR program, called “Tuungane,” implemented by the International Rescue Committee and CARE International in 1,250 villages throughout eastern Congo. The program had as a central goal to “improve the understanding and practice of democratic governance.”2 The goal of the program reflects broader normative goals shared by many international organizations and Western aid donors to promote a more recognizable democratic culture of governance. As a reflection of this goal, the program structurally sought to minimize the influence of traditional leaders in collective decision making. Implementers emphasized the normative desirability of the practices. The protocols of the program specifies aims of “[improving] good governance: practices of the transparency, accountability, representation, participatory management, inclusion of all.” Over a four year period, the program spent $46 million of development aid, reaching approximately 1,250 villages and a beneficiary population of approximately 1,780,000 people. A large share of this funding was used for facilitation and indirect costs, with only $16m, 35% of the total program costs, going directly towards infrastructure. These shares reflect the fact that the main focus of the intervention was institutional change, not the use of existing institutions to deploy development funds. Because this study focused more on learning about the social impacts (rather than the economic impacts) of the program, we focus on the first two of the overall four years, a period in which the institutional components were frontloaded and only $3.7m was spent on small-scale implementation. Figure 2 in the appendix (B) illustrates the timing of implementation (and data collection) across areas. The program followed well defined steps. First, populations were mobilized to townhall meetings, where the objectives, the implementation agencies, and the funding government were introduced. Second, the population was trained to participate in local democracy, which many had never experienced before (14% of the chiefs in our sample are elected through elections). Third, members of the village were encouraged to run for these elections.3Fourth, (private) elections were organized to elect a committee, whose task would be to manage the aid fund.4Fifth, committee members submitted a proposed spending plan for popular 2In 2007, in collaboration with the implementing partner, the research team developed hypotheses that took account of these goals. A broader set of hypotheses relating to behavioral outcomes were developed during implementation and prior to data collection. 3The only requirement to run was: “People may nominate themselves, but if they do so, they are required to have at least two other people support them.” Source: Tuungane protocols. 4For elections to be valid, at least 70% of the adult population had to vote. “At least 70 percent of the adult (over the age of 18) voting population must vote in order for elections to be valid. It is the responsibility of the Election Team and Tuungane to ensure adequate participation. This is the only way to legitimize 7 received something directly from the RAPID project. We calculate the value of each benefit and compute the standard deviation of the distributions that took place (in dollars) to represent the average difference in the amount received between two randomly selected villagers. Thus, we obtain a village level outcome that characterizes the village-level distribution of benefits. Finally, we measure the effect on the extent to which decisions reflect the preferences of the village chiefs versus those of the villagers. To do so, we compare the predictive power of the chief’s preferences to those of a random sample of five villagers. We obtain a village level measure that indicates the degree to which the chief’s private preferences (measured during Step A before the townhall) outperform the preferences of the five panel households who were also interviewed before the RAPID project in private (also collected during Step A before the townhall), at predicting RAPID project choice (observed at Step B).11 5.2.3 Measurement of Democratic Practices We examine democratic practice along three dimensions: participation, accountability, and transparency. Measurement of participation. First, during Step A, we count the number of villagers present in the town hall meeting of the RAPID project, and record the number of times that the average villager speaks in these meetings, as well as the dominance of men in such discussions. However, the patterns of public communication may not perfectly correspond to the actual inclusiveness of the process, for instance, if the most powerful individuals are less likely to talk in cases when town hall meetings are partly performative. We thus complement our measure of participation with the extent to which the RAPID project and the community committee that ends up being in charge of the funds are selected through participatory selection methods. That is, between Steps A and B, RAPID communities were required to select both a committee and a project as part of the terms of receiving funds. During Step B, enumerators conducted two focus groups simultaneously, one with members of the committee and a second with ordinary villagers, and coded the selection process as either electoral, through lottery, by consensus, imposed by the chief or elders, other, or unknown. We combine these four measures into one composite measure. Last, if the average villager is more likely to effectively participate, we should expect RAPID committees to have a broader representation of the population. We implemented an additional measure of participation: the composition of the RAPID committee. There was no constraint placed on 11We obtain similar results if we use data on RAPID project choice from Step D. Due to attrition in the latter we have more observations from Step B. 14 the composition of these committees other than size (at least 2 members and no more than 8). We create a composite measure based on the number of women, the number of men, the total size, and the share of women that make up the committee. Measurement of accountability. To measure accountability of elites, we first examine whether the community has put in place accountability mechanisms to control the actions of the committee in charge of the unconditional cash transfer funds. Was an external accountability measure (such as a distinct committee) put into place? Or was the committee required to report its actions to the community as a whole? During Step D, enumerators conducted focus groups with ordinary villagers, focus groups with at least two RAPID committee members, and interviews in private with the ten randomly selected respondents and two randomly selected committee members. We combine information from these four measures into one composite index. Next, we complement this measure of accountability by gathering information about people’s propensity to complain about the RAPID implementation. Specifically, during Step D, we asked the ten respondents in private to indicate whether or not they agreed with thirteen pre-selected complaints.12 We create an index of the average propensity of villagers to issue complaints. Measurement of transparency. First, we examine the degree to which the population is informed about the RAPID project. Specifically, we inform the entire community during the Step A town hall meeting that the village will receive at least $900 in unconditional cash transfers. However, about a week later, the RAPID project provides $1,000 to the RAPID village management committee. The committee thus learns about the actual amount transferred to the community in private. This introduces information asymmetry which allows us to measure the extent to which knowledge of the funding amount is subsequently shared by selected leaders to citizens. During Step D, we ask the ten randomly selected villagers to tell us the amount of the RAPID grant. We measure whether they report the correct amount. Second, we complement this measure with an incentivized behavioral measure. We randomly select two villagers in both RAPID and survey-only villages, and offer them monetary rewards to obtain their village’s school budget. We first measure the proportion of respondents who accept the task, and for those who refuse, why they do so. For those who accept and return, we compare the figures they report to the figures that we obtained as part of the direct interview with the school director. Finally, we measure the 12The process took too long; The organization (RAPID) did not behave well in villages; The projects selected were not the most important ones; The selected projects did not benefit a wide enough group; I had no real influence over the selection process; Disagreements were not well managed; The process was too complex; There was not enough information about the process; There was corruption (misuse of funds) in the village; The distribution of funds was not just; The project created divisions in the community; The RAPID committee was too influenced by the Chief; The RAPID committee did not represent our concerns. 15 quality of accounting. That is, RAPID committees were expected to keep an accounting form, and record the total amount made available for the RAPID project (out of $1,000) and what expenditures were made. During Step D, our auditors obtain copies of these accounting documents, and measure the presence of these documents and rate their quality and consistency following a precise metric that was developed ex-ante. 5.3 Estimation of the Treatment Effect To estimate the sample average treatment effect (Rubin, 1974) of taking part in the CDR program, we compare outcomes in Tuungane communities to outcomes in control communities, accounting for small differences in assignment propensities. Our core specification estimates: yijk =β0+β1Tj+νk+i(1) where iindicates the individual, jindicates the village cluster, and kthe lottery block. We include lottery block fixed effects, νk. The coefficient on Tjprovides our estimate of the treatment effect. In some specifications, we estimate: yijk =β0+β1Tj+β2Xi+β3XiTj+νk+i(2) This specification is used for the analysis of list experiments where the interest is in knowing whether the treatment increased the difference between long list and short list responses. In this case, the coefficient on XiTjcaptures the effect of the treatment effect on the difference between long and short lists. We also use this specification for the analysis of chiefly influence, where the interest is in knowing whether the treatment is associated with a smaller difference between the influence of chief and citizen preferences. We further clarify our strategy with respect to weighting, clustering of standard errors, controls, and multiple comparisons issues. Weighting. Because randomization blocks sometimes contain an odd number of clusters, some randomization blocks have slightly different assignment probabilities than those with even numbers (in which case 50% of units are selected). To account for this, we estimate the mean outcome in the treatment and control groups where we weight observations using the inverse of the treatment assignment probabilities (henceforth, inverse propensity weighting). With small deviations in assignment probabilities, this has a negligible effect on our estimates 16 (Angrist, 1998).13 We report results using sampling weights in the appendix (K), which produce the same conclusions. Controls. Controlling for randomization blocks can improve efficiency (Bruhn and McKenzie, 2009). Though not specified in our pre-analysis plan, in the main analysis, we present the results using block fixed effects. In the appendix (K.6) we show that our estimates are unchanged without them. Clustering. The CDR program is assigned at the village cluster level, j. For those analyses that make use of individual-level data, we cluster the standard errors at the level of treatment assignment. Multiple comparisons. For five of the fifteen outcomes, we use multiple measures and risk having multiple comparisons problems. Thus, we generate a summary of effects within each group of measures that are conceptually related, following the approach of Kling et al. (2007) and create a standardized index of measures in each group (see also Casey et al. (2013)). 6 Results We first present the effect of participating in the CDR program on the allocation of public funds, and then on the extent to which communities use more democratic governance practices. 6.1 Impacts on Public Fund Allocation Table 2 presents the effect of participating in the CDR program on the allocation of public funds.14 The “Control” column describes the estimated level for each measure in control communities. The subsequent column provides the size of the estimated effect of Tuungane, followed by our estimated standard error, the number of observations and the number of clusters in which data was collected. We now describe the key results. First, the effect on the share of the $1,000 grant that auditors were unable to account for is indistinguishable in villages that took part in the 13Technically we need to account for conditional heterogeneity in propensities given sampling into measurement. Thus, for example, if two units were randomly sampled from two blocks of size four and five, propensity weights would place weights of 2 on each of the four units in the first block and 2.5 on the treated units in the second block and 1.66 on the 2 control units in the second block. If, however, we sample only two of the three control units in the second block then reweighting is not needed to produce unbiased within-block estimates. Since we sample 280 out of 320 control areas into measurement we take account of this when producing propensity weights. 14See also Figure 3 in the appendix (F) for a graphical representation of these results. 17 CDR program from villages that had not taken part in the CDR program. On average, approximately 15% of the $1,000 could not be verified by the teams. There is no significant difference between Tuungane and control communities. The estimated effect is very small with a small standard error. This suggests little difference in fraudulent behavior across treatment groups, although this does not itself mean that resources that could be accounted for were used well. Second, the level of funds embezzlement in the RAPID project in villages that took part in the CDR program is indistinguishable from villages that did not. For the measure of embezzlement reported by households using the direct survey question, 15% of respondents in control communities report this to be a concern. Results are similar in Tuungane communities. When we examine instead the household reports of funds embezzlement based on the list experiment, we find again no statistically significant difference in Tuungane areas.15 Table 2: Effect on Public Fund Allocation Index Control Effect (se) N Clusters Financial Irregularities No 0.147 -0.006 (0.02) 394 394 Embezzlement (direct) No 0.147 -0.001 (0.018) 3623 411 Embezzlement (list experiment) No 0.462 -0.012 (0.066) 3676 411 Inequality of (Private) Benefits No 2.602 0.163 (0.495) 409 409 Dominance of Chief’s Preferences No 0.095 -0.019 (0.039) 2446 441 Notes: For “Embezzlement (list experiment)” and “Dominance of Chief’s Preferences” we estimate equation (2), and report estimates for β2in the Control column and estimates for β3in the Effect column, where Xis the sensitive item and the chief, respectively. All analyses employ inverse propensity weights, clustering of standard errors at the level of randomization clusters, and block fixed effects. Cluster column refers to the number of unique village clusters. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001. Third, villages that took part in the CDR program distribute the RAPID project funds with indistinguishable levels of inequality as villages that did not take part of the CDR 15For details on the number of clusters used for different analyses see appendix K.3. 18 program. On average, in control communities this standard deviation is around $2.6, and indistinguishable from Tuungane communities.16 Fourth, the preferences of the chief in villages that took part in the CDR program, and relative to the population’s preferences, are no less able to predict the RAPID project choice than in villages that did not take part of the CDR program. In control villages, the chief’s prior preferences are eight percentage points more likely than those of a randomly selected villager to coincide with actual RAPID project choice. However, the CDR program has no effect on the extent to which chief’s preferences outperform villagers’ preferences.17 In conclusion, exposure to the CDR program has no statistically significant effect on the choices made by communities during the RAPID project. We note a final observation on project outcomes. Discussions with the project funder and implementation agencies suggested expectations that in control areas most if not all of the $1,000 RAPID funds would be embezzled by traditional elites. Our data do not bear this out on any measure. The most pessimistic results come from the list experiment on embezzlement, where still more than half of all communities indicated no embezzlement. Other direct measures are even more positive on this front. 6.2 Democratic Practice Table 3 presents results related to democratic practice. Before discussing individual results, we note that there is a drop in observations when moving from our measures for participation to those for accountability and transparency. The reason is that the measures for participation build on data collected during Step A and B, while our measures for accountability and transparency build on data collected during Step D. Enumerator teams were expelled from Maniema province after most of the Step A data was collected, but before much of the Step D data was collected (see appendix K.3). Also note that our examination of democratic practices in a natural setting comes with implications for interpreting our estimand. We assess, for instance, whether the institutional intervention altered participation, which in turn may have affected choices. It is possible, however, that the intervention could have affected willingness to participate to prevent capture by elite but that this does not translate into greater participation because of the effectiveness of the threat. We measure effects on actual participation, not on propensity to participate. 16We note that in villages in which public goods were produced and no distributions with cash value were made this standard deviation is zero. 17The number of clusters is higher for this measure because it is based on data from Steps A and B. 19 6.2.1 Participation Table 3 shows that in control communities, on average, 132 adults participated in the public meeting, one more than in Tuungane communities; a very small difference which is not statistically significant. These meetings were attended by two enumerators, who recorded patterns of social interaction. The second row shows that, on average, 15 interventions are made per meeting, with only marginally fewer interventions in Tuungane communities. Furthermore, we find that in control communities men dominate the discussion, being responsible for 70% of the interventions. The third row shows that the patterns of male dominance of social interactions are indistinguishable across treatment and control.18 Next, we examine results related to the process of RAPID project and committee selection. Approximately 43% of committees and 31% of RAPID projects were coded as selected through election, and 71% of committees and 73% of the projects were selected through either election, lottery or consensus. Table 3 shows the composite measure, which by construction averages zero in control areas and has a standard deviation equal to one. We find that there is no evidence that participation in Tuungane leads to greater adoption of participatory processes in the selection of the committee or spending plans. The estimated effect is below one tenth of a standard deviation. Last, we examine the inclusiveness of the RAPID committee. On average, about one committee member in six was a woman (17% in control; 20% in treatment). The composite index — which includes the number of women, the number of men, the total size, and the share of women on the committee — shows that there is no statistically significant difference between Tuungane treatment and control communities.19 6.2.2 Accountability In the majority of villages, no mechanisms had been put in place to oversee the use of RAPID funding. However, 13% of respondents indicated that an external accountability measure (such as a distinct committee) had been put into place, and another 11% indicated that the committee had been required to report its actions to the community as a whole. As the composite measure in Table 3 indicates, Tuungane did not lead to a greater propensity to put accountability mechanisms into place. Second, we examine the propensity to complain as calculated by an index of the average propensity of villagers to issue complaints. Results in Table 3 suggest that levels of complaint 18We find similar results for dominance of the chief and elderly. 19Looking at the number of women and the share of women individually, we do find evidence that the Tuungane program had an impact. 20 Table 3: Effect on Democratic Practices Index Control Effect (se) N Clusters Participation Meeting Attendance No 132.394 -1.199 (6.297) 455 455 Interventions in Meeting No 14.697 -0.267 (0.483) 457 457 Dominance of Men in Discussion No 70.255 0.161 (1.36) 442 442 Participatory Selection Methods Yes 0.015 0.072 (0.073) 451 451 Committee Composition Yes 0.033 0.099 (0.078) 452 452 Accountability Accountability Mechanisms Yes 0.007 -0.036 (0.094) 414 413 Private Complaints Yes 0.015 -0.01 (0.052) 3658 411 Transparency Knowledge of Project Amount No 37.965 0.697 (2.384) 3699 411 Willingness to Seek Information No 39.161 2.399 (2.335) 1407 411 Quality of Accounting Yes -0.026 0.011 (0.084) 399 399 Notes: Outcome measures in rows 4 to 7 and row 10 are indices and by construction have zero average in control areas. Deviations from 0 in the control column may arise because we report the weighted average of block average outcomes in control areas, which differs marginally from the overall average. All analyses employ inverse propensity weights, clustering of standard errors at the level of randomization clusters, and block fixed effects. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 are no higher in Tuungane areas than in control. Note that it could be that they are no higher because RAPID project management was better, although, as we saw above, there is little evidence to support this.20 20We have also conducted analysis of whether complaints are greater conditional on mismanagement. This analysis suggests a positive effect of the program. However, we note that this test was not registered nor is it identified, since mismanagement is potentially endogenous. 21 6.2.3 Transparency To assess effects on transparency we first examine people’s knowledge regarding the grant amount. Table 3 shows that, on average, 38% of all respondents report the correct answer of $1,000. However, we find no evidence that there is a difference between treatment and control communities. Second, we examine the effect on the willingness to seek information relevant to the public. Table 3 shows that approximately 39% of those in control communities were willing to seek information (receiving one dollar for the attempt, and an additional dollar upon success). The people that refused gave various reasons: that it is not appropriate to ask for this information (76), that the respondent did not have time (75), that the exercise is strange to them (40), that the husband of the respondent refuses or would refuse the collection of this information (9), and other reasons (192). This suggests that individuals do not feel they have rights to access basic financial information. There is no significant difference between treatment and control communities. Finally, we examine the effects on quality of accounting. We find that, on average, in 83% of the villages, the committee had their accounting form present upon arrival of the audit team during Step D. Approximately 80% of the funds were correctly accounted for as calculated by the RAPID Committee (and 80% when calculated by the audit teams). In addition, 58% of the money the committee made available for the RAPID project (of the $1,000) was justified by receipts, and 47% was justified with receipts deemed credible by the auditing team. Table 3 presents the composite index taking these individual measures into account. We do not find evidence of an impact of Tuungane on the existence and quality of accounting. In summary, we find no evidence that the intervention induced democratic practices. 7 Is the Null Due to Failures in Design? Our null results might reflect a weak treatment effect, but they might also reflect weaknesses in our research design. We explore three possible reasons that could in principle produce false null results: spillover bias, differential social desirability biases, and low statistical power. In addition, we consider the possibility that results reflect a bias due to a short term elite response to the intervention (see appendix I). We also explore survey-based information that addresses the concern that the intervention itself was a poor case for finding evidence of effects, or that the intervention was poorly implemented (appendix J). Finally, we explore 22 concerns related to data missingness, compliance, treatment heterogeneity and specification biases (appendix K). 7.1 Spillovers It is possible that Tuungane produced positive effects beyond treatment communities and that these positive spillovers bias our estimates of effects downwards. To address this concern we take account of indirect village exposure to treatment. We define an “x-km indirect effect” as the effect of being within xkilometers of a Tuungane village that is part of another cluster of villages.21 Because of some data missingness our results assess the effects of being close to a treatment village for which we have location data. We ignore this detail in what follows in light of the small number of units with missing data (we have GPS locations for a total of 1,020 of the 1,120 villages). The propensity of being exposed to such a treatment effect depends not just on the random assignment of units to treatment but also on the location of any given unit with respect to others. We make use of the random assignment of Tuungane to recover these propensities, since they are determined by our original randomization. To calculate these propensities we replicate the random assignment to Tuungane to obtain 5,000 possible assignments of all units to treatment and control, employing the same scheme as used in the original randomization. We then assess, for each unit, the probability of receiving direct treatment, indirect treatment, and each combination of these. To avoid instability arising from large weights, we limit the analysis to villages that have at least a 10% to 90% probability of being in each of these groups for a given value of x. We then generate estimates of treatment effects by comparing outcomes in each combination of conditions with inverse propensity weighting using the known propensity for each unit of being in each condition. We test the sharp null of no effects using a randomization inference procedure (Fisher, 1935).22 We conduct our analysis for both a 5km radius spillover treatment and a 20km radius 21Note that for the spillover analysis missing data affects both the set of units in the study but also the measures of exposure to spillovers. The strategy we use to assess the presence of spillovers is design-based in the sense that it uses information on the probability of exposure to spillovers that can be calculated from the assignment strategy. See Gerber and Green (2012), Chapter 8, for more details. We emphasize however that this does not mean that we do not depend upon a substantive model: rather we require a type of SUTVA violation such that units not depend upon the treatment condition of other units beyond those described by the “x-km indirect exposure (Aronow and Samii, 2013). 22That is, for each of the 5,000 re-assignments to Tuungane we calculate the estimated effect of each treatment type for each outcome of interest. Combined, these estimated effects produce a reference distribution of model root mean squared error (RMSE) under the sharp null. We compare the actual root mean squared error to this distribution and estimate how likely it is we would have obtained results as strong or stronger than the actual RMSE under the sharp null. See also: Barrios et al. (2012). 23 Cantoni, D. (2012). Adopting a New Religion: The Case of Protestantism in 16th Century Germany. Technical Report 560. Casey, C. (2018). Radical decentralization: Does community driven development work? Annual Review of Economics, 10:139–163. Casey, K., Glennerster, R., and Miguel, E. (2013). Reshaping Institutions: Evidence on Aid Impacts using a Preanalysis Plan. Quarterly Journal of Economics, 127(4):1755–1812. Chattopadhyay, R. and Duflo, E. (2004). Women as Policy Makers: Evidence from a Randomized Policy Experiment in India. Econometrica, 72(5):1409–1443. Chwe, M. S.-Y. (2000). Communication and Coordination in Social Networks. Review of Economic Studies, 67(1):1–16. Fearon, J. D., Humphreys, M., and Weinstein, J. M. (2009). Can Development Aid Contribute to Social Cohesion after Civil War? Evidence from a Field Experiment in PostConflict Liberia. American Economic Review: Papers & Proceedings, 99(2):287–291. Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, London. Fujiwara, T. (2015). Voting technology, political responsiveness, and infant health: Evidence from brazil. Econometrica, 83(2):423–464. Gerber, A. S. and Green, D. P. (2012). Field Experiments: Design, Analysis, and Interpretation. W.W. Norton, New York City. Giuliano, P. and Nunn, N. (2017). Understanding cultural persistence and change. Gonz´alez de Lara, Y., Greif, A., and Jha, S. (2008). The Administrative Foundations of Self-Enforcing Constitutions. American Economic Review, 98(2):105–109. Greif, A. (1994). Cultural beliefs and the organization of society: A historical and theoretical reflection on collectivist and individualist societies. Journal of Political Economy, 102(5):912–950. Greif, A. and Laitin, D. D. (2004). A Theory of Endogenous Institutional Change. American Political Science Review, 98(4):633–652. Grossman, G. (2014). Do Selection Rules Affect Leader Responsiveness? Evidence from Rural Uganda. Quarterly Journal of Political Science, 9(1):1–44. 30 Grossman, H. I. and Kim, M. (1995). Swords or Plowshares? A Theory of the Security of Claims to Property. Journal of Political Economy, 103(6):1275–1288. Guggenheim, S., Wiranto, T., Prasta, Y., and Wong, S. (2004). Indonesias kecamatan development program: A large-scale use of community development to reduce poverty. In Scaling Up Poverty Reduction: A Global Learning Process and Conference. Shanghai, May, pages 25–27. Haley, K. J. and Fessler, D. M. (2005). Nobody’s Watching? Subtle Cues Affect Generosity in an Anonymous Economic Game. Evolution and Human Behavior, 26(3):245–256. Hamilton, W. D. (1964). The Genetical Evolution of Social Behavior. Journal of Theoretical Biology, 7(1):1–16. Hartford, T. (2012). Cash Delivery on the World’s Poorest, https://www.ft.com/content/7ce1b356-f6f9-11e1-9dff-00144feabdc0. Financial Times. Hartford, T. (2014). One of the largest ever randomized control trials... results are in. http://timharford.com/2012/09/one-of-the-worlds-largest-ever-randomised-trials-theresults-are-in/. Technical report. Herbst, J. (2014). States and Power in Africa. Princeton University Press, Princeton. Hoffmann, K. (2014). Ethnogovernmentality: The Making of Ethnic Territories and Subjects in Eastern Congo: Ph.D. Dissertation. Doctoral School of Society and Globalisation, Department of Society and Globalisation, Roskilde University. Hoffmann, K., Vlassenroot, K., and Marchais, G. (2016). Taxation, stateness and armed groups: Public authority and resource extraction in eastern congo. Development and Change, 47(6):1434–1456. Humphreys, M., Sanchez de la Sierra, R., and Van der Windt, P. (2012). Social and Economic Impacts of Tuungane: Final Report on the Effects of a Community Driven Reconstruction Program in Eastern Democratic Republic of Congo. Humphreys, M., Sanchez de la Sierra, R., and Van der Windt, P. (2013). Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration. Political Analysis, 21(1):1–20. King, E. and Samii, C. D. (2014). Fast-Track Institution Building in Conflict-Affected Countries? Insights from Recent Field Experiments. World Development, 64(1):740–760. 31 Kling, J. R., Liebman, J. B., and Katz, L. F. (2007). Experimental Analysis of Neighborhood Effects. Econometrica, 75(1):83–119. Lowes, S., Nunn, N., Robinson, J. A., and Weigel, J. (2017). The evolution of culture and institutions: Evidence from the kuba kingdom. Econometrica, 85(4):1065–1091. Mamdani, M. (1996). Citizen and Subject: Contemporary Africa and the Legacy of Late Colonialism. Princeton University Press, Princeton. Mansuri, G. and Rao, V. (2013). Localizing Development: Does Participation Work? World Bank Policy Report. Martinez-Bravo, M., Padro i Miguel, G., Qian, N., and Yao, Y. (2011). Do Local Elections in Non-Democracies Increase Accountability? Evidence from Rural China. NBER Working paper, 16948. Mead, M. (1968). New Lives for Old: Cultural Transformation -Manus, 1928-1953. Mehta, J., Starmer, C., and Sugden, R. (1992). An Experimental Investigation of Focal Points in Coordination and Bargaining: Some Preliminary Results. In Decision Making under Risk and Uncertainty, pages 211–219. Springer. Newbury, D. (1991). Kings and clans: Ijwi Island and the Lake Kivu Rift, 1780-1840. University of Wisconsin Press. Nunn, N. and Qian, N. (2014). US Food Aid and Civil Conflict. American Economic Review, 104(6):1630–1666. Paluck, E. L. (2009). Reducing Intergroup Prejudice and Conflict Using the Media: A Field Experiment in Rwanda. Journal of Personality and Social Psychology, 96(3):574–87. Paluck, E. L. and Green, D. P. (2009). Deference, Dissent, and Dispute Resolution: An Experimental Intervention Using Mass Media to Change Norms and Behavior in Rwanda. American Political Science Review, 103(04):622–644. Raeymaekers, T. (2014). Violent Capitalism and Hybrid Identity in the Eastern Congo: Power to the Margins. Ross, M. L. (2001). Does Oil Hinder Democracy? World Politics, 53(3):325–361. Rubin, D. B. (1974). Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies. Journal of Educational Psychology, 66(5):688–701. 32 Sanchez de la Sierra, R. (2017). On the origins of the state: Stationary bandits and taxation in eastern congo. Unpublished. Shayo, M. (2009). A Model of Social Identity with an Application to Political Economy: Nation, Class, and Redistribution. American Political Science Review, 103(02):147–174. Shepsle, K. A. (2006). Rational Choice Institutionalism. The Oxford Handbook of Political Institutions, pages 23–38. Sokoloff, K. L. and Engerman, S. L. (2000). History Lessons Institutions, Factor Endowments, and Paths of Development in the New World. Journal of Economic Perspectives, 14(3):217–232. Stearns, J., Verweijen, J., and Baaz, M. (2013). The National Army and Armed Groups in the Eastern Congo: Untangling the Gordian Knot of Insecurity. Usalama Project. Rift Valley Institute. Stearns, J. and Vogel, C. (2015). The Landscape of Armed Groups in Eastern Congo. Center on International Cooperation at New York University, New York. Voigtlander, N. and Voth, H.-J. (2012). Persecution perpetuated: The medieval origins of anti-semitic violence in nazi germany. The Quarterly Journal of Economics, 127(3):1339. Young, H. P. (2001). Individual Strategy and Social Structure: An Evolutionary Theory of Institutions. Princeton University Press, Princeton. 33 Appendix 34 A Relation to Theoretical Literature Our study also relates to theoretical literature that sheds light on logics through which exposure to democratic practices might alter political institutions. The sources of institutional change, and thus, how external influence might affect institutions, depend on how institutions are conceptualized. Shepsle (2006) distinguishes between two prevalent concepts of institutions (see also Greif and Laitin (2004)). In one, institutions are the rules of the game, with enforcement of those rules guaranteed outside of the game — where “rules” refers to the mapping from actions to payoffs. In the second, institutions are the equilibria of a game, which endogenously constrain behavior. In the following section, we use a simple game to describe two intervention strategies that produce observationally equivalent behavior. One strategy, Strategy A, affects behavior by altering expectations but without changing the primitives of the game; the other, Strategy B, alters behavior by altering primitives — the mapping from actions to outcomes. Both strategies yield identical outcomes and the equilibrium payoffs are the same. Equilibria shifts arising from Strategy Aare akin to “poverty trap” arguments in the economic growth literature. These two conceptualizations of institutions map onto two strands of research on institutional change. On the one hand, many empirical studies examine the effects of changing rules of the game alone, such as electoral rules (see Chattopadhyay and Duflo (2004) and Fujiwara (2015)). On the other hand, long-run accounts often share the view of institutions as equilibria (Boyd and Richerson, 2002). Young (2001), for example, provides an account of social institutions as patterns of behavior that may exhibit large variations across space and time without any change to primitives.28 Variation in the quality of property rights regimes, norms of fairness, or tolerance for less accountable governments can also be observed across societies that share common characteristics and for which variation is potentially attributable to equilibrium selection logics (Grossman and Kim, 1995; Binmore, 1998; Young, 2001; Chwe, 2000; Bidner and Francois, 2013; Acemoglu et al., 2011). Different belief systems have been shown to support different equilibrium forms of social organization (Greif, 1994). This is similar to the view of constitutions as self-enforcing equilibria among administrators that coordinate to constrain the power of the ruler (Gonz´alez de Lara et al., 2008). Since in failed states, rules are often hard to change through law, many interventions seek to shift norms and practices, or expectations of behavior, aiming to induce a better equilibrium. However, explanations of the transition to a new equilibrium (the theory underlying Strategy A) emerged from outside classical game theory. Both logics formally underpin the 28Seemingly deep social structures coupled with policy choices can obtain as equilibria in environments where very different equilibria also obtain, supported by the same fundamentals (Shayo, 2009). 35 rationale of external interventions like the one we study here. On the one hand, research in the lab supports the importance of leaders and moral authorities who can influence beliefs. Equilibrium-irrelevant interventions (such as labeling options or framing the context), which leaders often can change, can change the focality of equilibria, thus leading individuals to coordinate on new equilibria (Mehta et al., 1992).29 On the other hand, research in the area of cultural evolution shows that while cultural norms often persist through generations (Alesina et al., 2013; Voigtlander and Voth, 2012), adoption can also occur relatively fast (Cantoni, 2012; Mead, 1968). The literature suggests that humans’ strong tendency to imitate practices perceived to be more successful can explain the evolution of culture (Boyd and Richerson, 2002). Inter-generational transmission of culture co-exists with individual optimization and the transmission of culture across individuals and across groups (Giuliano and Nunn, 2017). But, then, when do societies adopt new practices?30 According to this research, the perceived success of such practices, as well as the degree of prestige of the group who practices it are important determinants of imitation and adoption (Aoki and Feldman, 1987).31 A.1 Institutional Logics Consider a simple game in which two players, Strong (S) and Weak (W), can each decide in each of an infinite number of periods whether to produce using a default technology (D) or a cooperative technology (C). Say each period decision resembles a prisoner’s dilemma. If both use the cooperative technology they produce output worth 1 unit. If both stay with the default technology their yield is dj=.5 for j∈ {S, W}. If one uses the default technology while the other attempts to use the cooperative technology on her own the first receives 29Bidner and Francois (2013) provide a more developed approach in the context of a model of accountability relations in which changes in norms occur endogenously following particular sequences of actions by leaders. Similar results would obtain from the limited rationality models in Young (2001), where expectations are based on observation of past actions and equilibria could change following a period of deviations induced exogenously. 30Related applications include “social norms marketing” in Paluck and Green (2009), and Paluck (2009). 31The view that primitives determine institutional change has gained traction because the changes in the primitives are easier to measure, track and, unlike practices, they are easier to articulate in economic theory (Acemoglu et al., 2001; Sokoloff and Engerman, 2000). Sokoloff and Engerman (2000) in their study of the role of institutions emphasize the importance of factor endowments in determining structural inequality; indeed they highlight the “clear implication that institutions should not be presumed to be exogenous.” In the account provided by Herbst (2014), institutional variations in state capacity and responsiveness also reflect more fundamental features, notably agricultural technology and population densities. Others emphasize access to resources, such as subsoil resources (Ross, 2001) or aid (Nunn and Qian, 2014). Yet, it remains an empirical question whether culture, practices, and equilibrium selection can also play an important role in explaining the observed variation of institutional change. 36 free-rider yield fj∈(1,2) while the second receives 0. In addition, players can make cash transfers to each other (assuming utility is linear in income, we treat utility as transferable). Baseline equilibrium. With sufficient patience, the following is a subgame perfect equilibrium of this game: both players cooperate every period, each producing .5 units of value more than they would over the returns using the default technology. Player Wthen transfers .4 units of value to player S, and players end the round with payoffs of .6 for W and 1.4 for S. If in any period a player plays Dor the appropriate transfer is not made, then all players play Din every subsequent period. In this equilibrium Sextracts 80% of what Wproduced over and above what she would have gained had they both played D. Following Greif and Laitin (2004) this equilibrium is the institution, it is sustained by equilibrium expectations of players that cooperation will only be sustained if Wmakes large transfers to S. In this case we might think of the political part of the institution as the 80% tax rate imposed on W. Suppose now a third party views this equilibrium as exploitative and seeks to change outcomes. Consider two strategies they might employ. Strategy A.The first strategy seeks to improve the lot of Wby changing the equilibrium. Leaving the game intact, the third party proposes that the surplus be divided more equally, perhaps proposing that Wonly transfers half as much each period to R, leaving Wand Swith 1.2 and 0.8 respectively. The strategy is motivated by the observation that a 40% tax regime (on surplus) can also be sustained in equilibrium and so if players adopt the right expectations the new transfers will be self-enforcing. This intervention is a purely institutional intervention: it focuses on expectations and leaves the underlying game unchanged. Strategy B.Consider now a second intervention in which the third party guarantees Wa return of dW= 0.75 instead of dW= 0.5 in the event of cooperation failure. This is a structural change and has a real effect on W’s bargaining position. It means that Wcan now do better playing Din all periods and giving up cooperation with S. Both will still do better under some cooperative arrangement however. Say in the event of cooperation, Scontinued to extract 80% of W’s surplus. Then she would now force a transfer of .25 ×.8 = .2 and so Wwould be left with 0.8. Strategy Bproduces the same outcome (0.8,1.2) as achieved by Strategy Abut does so without requiring a change in the approach used by the players to divide the surplus. Moreover the behaviors on the equilibrium path following the two interventions are the same – both players play C, each earns 1 unit and Wtransfers .2 units to S. The effect of Strategy Bhowever is not due to changes in the equilibrium selected but to a change in the underlying 37 game (albeit one that matters only off the equilibrium path). 38 B Timing of Intervention and Measurement Figure 2: Timeline of Implementation HAUT KATANGA Jun 2007 Feb 2009 Oct 2010 Jun 2012 KAPONDA LUFIRA KAFIRA BAKUNDA BUKANDA SOURCE DF CONGO KINAMA KISAMAMBA LWAPULA BASANGA MANIEMA Jun 2007 Feb 2009 Oct 2010 Jun 2012 BEIA BASONGOLA AMBWE SOUTH KIVU Jun 2007 Feb 2009 Oct 2010 Jun 2012 BUHAVU NGWESHE WAMUZIMU BASILE BAFULIRO LWINDI TANGANYIKA Jun 2007 Feb 2009 Oct 2010 Jun 2012 NYEMBO BENZE BALUBA YAMBULA LUKUSWA MUHONA BAYASHI LUBUNDA LWELA LUVUNGUY BASONGE NKUVU MUNONO TUMBWE BENAMAMBWE Notes: Thin black lines indicate length of the Tuungane CDR program per chiefdom. Thick line indicates the first phase, which is the one we study here. Shorter, red lines indicate the period of measurement in that chiefdom. Source: Authors’ drawing. 39 Table 6: Spillovers at 5km Direct Indirect RMSE (p) N Spillovers at 5km Financial Irregularities 0.07 -0.04 0.43 0.94 156 Embezzlement (direct) 0.11 -0.04 0.42 0.83 163 Embezzlement (list experiment) 0.1 0.1 1.27 0.52 163 Inequality of (Private) Benefits 0.89 0.41 8.22 0.22 163 Dominance of Chief’s Preferences -0.05 0.09 0.79 0.5 157 Participation Meeting Attendance -14.65 1.07 138.85 0.77 171 Interventions in Meeting 0.28 0.45 9.45 0.4 172 Dominance of Men in Discussion 1.17 0.23 29.41 0.8 169 Participatory Selection Methods 0.26 0.16 2 0.73 170 Committee Composition 0.23 -0.16 1.85 0.54 170 Accountability Accountability Mechanisms -0.02 -0.06 1.95 0.81 164 Private Complaints 0.34 -0.03 1.41 0.89 163 Transparency Knowledge of Project Amount -2.28 2.63 52.02 0.92 163 Willingness to Seek Information 7.1 -6.43 81.6 0.94 301 Quality of Accounting -0.22 -0.13 2.03 0.59 157 Notes: Spillover effects estimated using a regression model of the form Y=αDirect +βIndirect +γDirect ×Indirect where both the direct and indirect maesures are normalized to have zero means. Average direct and indirect effects are then given by αand β. RMSE is used as a test statistic for the randomization infernence and the pvalue reports the probability of such a low RMSE under the sharp null of no effects. 46 Table 7: Spillovers at 20km Direct Indirect RMSE (p) N Spillovers at 20km Financial Irregularities -0.02 0.01 0.29 0.24 119 Embezzlement (direct) 0.03 0.03 0.32 0.54 126 Embezzlement (list experiment) -0.02 -0.16 1.01 0.49 126 Inequality of (Private) Benefits -0.16 -0.55 5.99 0.89 126 Dominance of Chief’s Preferences 0.12 -0.03 0.74 0.57 120 Participation Meeting Attendance 6.53 -42.05 146.56 0.98 141 Interventions in Meeting -0.22 0.79 9.54 0.61 141 Dominance of Men in Discussion 0.28 0.84 29.67 0.91 133 Participatory Selection Methods -0.12 -0.01 1.69 0.73 141 Committee Composition 0.41 0.34 1.48 0.53 142 Accountability Accountability Mechanisms -0.1 -0.42 1.77 0.49 126 Private Complaints 0.02 0 0.84 0.5 126 Transparency Knowledge of Project Amount -4.46 -1.3 41.61 0.38 126 Willingness to Seek Information -2.14 11.36 68.11 0.2 241 Quality of Accounting 0.06 -0.14 1.33 0.3 120 Notes: Spillover effects estimated using a regression model of the form Y=αDirect +βIndirect +γDirect ×Indirect where both the direct and indirect maesures are normalized to have zero means. Average direct and indirect effects are then given by αand β. RMSE is used as a test statistic for the randomization infernence and the pvalue reports the probability of such a low RMSE under the sharp null of no effects. 47 H Differential Desirability This section presents the social desirability test. Table 8 presents the difference in the propensity to respond yes to the question ”do you agree with the view that elections are the best way to choose community representatives to serve in positions that requires technical expertise?” as a function of the prompt that was given prior to the question. The table shows that, while the prompt has a significant effect on the proportion of individuals who answer yes – indicating social desirability bias – such effect is indistinguishable in treatment and control. Table 8: Social Desirability Test Positive prompt Negative prompt Difference (se) Control 0.642 0.843 0.201 0.02 Tuungane 0.65 0.859 0.21 0.021 Difference 0.007 0.016 0.009 (se) 0.025 0.019 0.029 Notes: N=3,802. Share of individuals answering ‘yes’ to the question “Do you agree with the view that elections are the best way to choose community representatives to serve in positions that require technical expertise?” ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001. 48 I Elite Backlash Against Loss in Power Since village chiefs were actively excluded from the Tuungane program, they might have had incentives to seek compensation during RAPID. In this case the null result could reflect unusually strong incentives for traditional leaders to engage in capture in treatment groups, coupled with strong restraints induced by bottom up pressures following the intervention. In Section 6 we found that the implemented RAPID projects (obtained during Step D) coincide better with the stated preferences (taken during Step A before the village meeting) of the chief than those of the villagers. We interpreted this as possible evidence for chief dominance. To explore whether the chief captured the RAPID process, and especially so in Tuungane areas, we investigate whether in Tuungane areas members of the RAPID committee are more closely related to the village chief. To measure network proximity, we collected detailed friendship and kinship data among randomly selected villagers, which includes their relationship with all committee members as well as with the chief. We also collected detailed friendship and kinship data among all committee members, which includes their relationship with the village chief. We then create a measure of family connection to the committee using the Hamilton index.32 We find that neither the population nor the chief are closely related to RAPID committee members. The average score on our index for the population is 3.49%, while the score for the village chief is 4.45%. This difference is statistically significant, and amounts to the chief adding a first cousin to the RAPID committee.33 We find no difference in this kinship proximity between Tuungane and non-Tuungane areas, however. Other measures also confirm that the chief did not disproportionately dominate RAPID procedures in Tuungane areas. During Step B, our enumeration team led focus groups with ordinary villagers to learn whether the process of RAPID committee and RAPID project selection was electoral, by lottery, by consensus, imposed by the chief, by elders, other or unknown. Very few people (around 5%) find that the chief imposed RAPID project selection or committee member selection and equally so in treatment and control areas. Finally, during Step D we directly ask individuals whether the RAPID committee was controlled by the chief. Around 26% of the 2,514 individuals answer in the affirmative. However, from a menu of thirteen complaints less than 5% of the respondent find this to be the most important complaint. Moreover, there are no differences in reporting across treatment and control 32The Hamilton index measures the biologic relatedness between two individuals: for a parent-offspring or full sibling relationship this index is 50%, for an aunt/uncle or nephew/niece relationship this is 25%, etc. See: Hamilton (1964). Applied to the group, if for example, two members of the RAPID committee, out of the five, are children of the chief and one is a nephew, the chief’s Hamilton score is 25%. 33Note, however that in almost 63% of the villages have no relationship at all to the chief. 49 areas. We thus conclude that the null result reported in this paper does not reflect chiefs’ response to Tuungane. 50 J A Bad Case? One possible explanation for weak evidence of effects is that this was simply a weak intervention: it was poorly implemented or not typical of the kind that researchers or policy makers expect to generate strong effects. We have direct evidence, however, on the extent to which development funders and implementers supporting this CDR program expected that it would produce strong effects. To find out, and prior to launching our endline data collection, we ran a small survey with the population of regional program implementers and CDR program directors (12 respondents) as well as a (convenience) sample of seven researchers working in eastern Congo and Rwanda on related issues. The survey simply elicited beliefs regarding likely impacts on each of the outcomes in different categories. It was not incentivized. The responses showed variation from item to item – which suggests that respondents were not simple optimists. Two thirds of program implementation respondents reported that they thought it “improbable” that beneficiaries would allocate more time to income generating activities; none thought it very likely that household incomes would increase. Yet, all but one thought it possible or very likely that there would be improvements in each of three distinct dimensions of governance outcomes. Half thought it very likely that villages would manage CDR program in a more transparent and equitable way. Researchers were more optimistic about effects on participation, but considerably more skeptical that traditional leaders would become more accountable (most researchers reported that they would not). Access to this information is valuable for the simple reason that it was formed prior to data gathering. If the weakness of the intervention seems obvious after the results are in, our information on priors supports the idea that the lessons may extend nevertheless to cases that are currently believed to be models. Overall, prior beliefs reflect confidence that CDR is an effective model. 51 K Robustness In the text we discuss concerns related to spillovers and to social desirability biases. Here we describe issues related to attrition and data missingness, noncompliance, treatment heterogeneity, and specification sensitivity. K.1 Sampling weights The main analyses presented focus on sample average treatment effects. When we collect household-level data, we sample ten households in each village. Failing to account for heterogeneous sampling probabilities would result in a biased estimate of the average household in the population of interest. Similarly, in each household, we interview one respondent randomly selected within the household. Similarly, failing to account for heterogeneous sampling probabilities within households would produce a biased estimate of the population average. In Table 10, we thus employ sampling weights, derived from the sampling procedure within village across households, and within households, to account for differences in sampling probabilities across individuals in the population. These obtain the population average treatment effect (as opposed to the sampling average treatment effect in the main result). The results are unchanged. K.2 Challenges to Implementation The data collection effort was a very large undertaking implemented by almost 100 surveyors and their corresponding supervision and management structure over the course of over a year in a region the size of France but without any of the infrastructure. We provide a brief account of the logistics of data undertaking. Research Teams. Multiple teams were engaged in implementing RAPID and gathering outcome data. Each province had two teams for step A, which each consisted of a RAPID project facilitator and an assistant. A teams were responsible for introducing the RAPID project to the village chief and to the village during a general assembly and for conducting a set of surveys (to be discussed in more detail below). One B team in each area visited the villages a week after Team A. These teams were responsible for meeting the committee and conducted focus groups to learn how both the committee and the RAPID project was chosen. These teams included the Provincial Supervisor who had a satellite phone in order to call the IRC or CARE International headquarters, so that a C team could visit the village to distribute the RAPID project funds. During Step C, disbursement was done by IRC or 52 CARE staff but without identifying themselves as such. Approximately 48 days later, after the implementation of the RAPID project, both RAPID and non-RAPID villages were visited by D Teams. D teams in RAPID villages included three enumerators and one auditor. The latter did a detailed investigation into how RAPID grants were spent and, where applicable, located beneficiary populations. The survey-only villages consisted of only two enumerators. In addition to A, B, C and D Teams each province had two Super-assistants: one responsible for Step A to Step C, and one for Step D. Super-assistants visited teams to collect and backup data, photos and GPS coordinates, and ensured quality control. These staff were hired and directed by leads at the Universities of Bukavu and Lubumbashi. Finally, there were two Regional Evaluation Coordinators, hired by the IRC: one based in Bukavu (South Kivu) and one in Lubumbashi (Haut Katanga). They were responsible for supervision of implementation, and monitoring the data collection and its quality. These Coordinators were in daily contact with the Columbia University research team and worked closely together with the Research and Evaluation Coordinator of the IRC. Between June and December 2010 two of the authors were based in Congo to launch the RAPID project. Security. The area where the research took place is marked by high levels of insecurity, especially in South Kivu. The security of the teams was a major concern throughout and teams were not allowed to visit a village before receiving security clearance from the IRC’s security team. The latter had contact with the major actors such as the United Nations peacekeeping forces, the DRC government and others. Despite the precautions undertaken we did encounter some security issues: 31 villages were not visited due to security risks; one team was ambushed and had to hand over their equipment; and one IRC staff member was abducted (and subsequently released unharmed) during the implementation of Step C. In particularly risky areas of South Kivu, Step C was undertaken through accounts in local credit offices (COOPECs) rather than having cash delivered by a field agent. The challenges to implement a data collection exercise in this area account for most of the missing responses, which we describe in detail next. K.3 Attrition and Missing Responses A first threat to validity stems from missing responses. Our study involves a complex collection of units – individuals, grouped into settlements, grouped into villages, grouped into village clusters, grouped into lottery blocks. Attrition in this project took place at different levels, with different implications. The most important reason for attrition, which took place due to expulsion of our teams from the Maniema province, is at the block level arising from political instability. This attrition is by design balanced for treatment and 53 control groups. Other attrition takes place at the level of individual villages or individual respondents. We describe various threats arising from attrition in turn. Figure 5: Diagram Summarizing the Organization of Units, Assignment and Measurement Strategies, and Sources of Attrition. Based on CONSORT flow chart. Abbreviations: LLUs = Lowest Level Units (settlements); HK = Haut Katanga, MN = Maniema, SK = Sud Kivu, TG = Tanganyika. CDR Assignment 3,000 settlements (“LLUs”), grouped into: - 1,250 villages - 600 village clusters - 83 lottery blocks CDR random assignment to clusters within blocks RAPID Assignment 560 of 600 clusters sampled for RAPID lottery and survey measurement Village clusters allocated to CDR program (n= 280) LLUs allocated to RAPID (n=280) One LLU per cluster receives RAPID RAPID: S tep A Village clusters allocated to CDR control (n= 320) “Survey-only” LLUs (n= 280) One LLU per cluster receives survey-only “Survey-only” LLUs (n= 280) One LLU per cluster receives survey-only LLUs allocated to RAPID (n=280) One LLU per cluster receives RAPID Targeted households: 280*5=1,400 LLUs visited=229 Household surveys collected=1,111 LLUs visited=208 LLU loss (n=72): HK=0, MN=65, SK=3, TG=4 Target: Panel households: 280*5=1,400 Additional households: 280*5=1,400 Collect info measure: 280*2=560 Collected: Panel (n=947) Additional (n=981) Info (n=354) LLUs visited=200 LLU loss (n=80): HK=6, MN=65, SK=2, TG=7 Target: Households: 280*5=1,400 Collect info measure: 280*2=560 Collected: Surveys (n=946) Info (n=342) RAPID: Step D Targeted households: 280*5=1,400 LLUs visited=228 Household surveys collected=1,103 LLUs visited=205 LLU loss (n=75): HK=1, MN=65, SK=3, TG=6 Target: Panel households: 280*5=1,400 Additional household surveys: 280*5=1,400 Collect info measure: 280*2=560 Collected: Panel (n=916) Additional (n=971) Info (n=350) LLUs visited=203 LLU loss (n=77): HK=3, MN=64, SK=3, TG=7 Target: Households: 280*5=1,400 Collect info measure: 280*2=560 Collected: Surveys (n=983) Info (n=366) LLUs visited=228 RAPID: Step B LLUs visited=226 RAPID: Step C LLUs received grant =228 LLUs received grant =226 The study was designed to gather data in a sample of 1,120 villages, half of which were randomly selected for the RAPID project. Different targets were set for different items but the most common data (the household survey) were to be gathered for ten households in RAPID villages and five households in survey-only villages. Given that there were 560 54 RAPID villages and 560 non RAPID villages this makes a total of 8,400 households (for some items gathered only in RAPID or only in survey-only areas, the targets were 2,800). However, the survey teams successfully collected final (Step D) in only 816 villages (413 RAPID and 403 survey-only villages) and from 5,744 households. The full complement of targeted data was not gathered for a number of reasons. Figure 5 presents a CONSORT-style flow chart with the details on the number of villages targeted and visited, and surveys targeted and collected. The figure shows that the most significant site of missing data is Maniema province. Political tensions in the run up to the November 2012 presidential elections led to the expulsion of the Maniema teams shortly after the launch of Step D. This led to the loss of 130 RAPID villages and 129 survey-only villages for all measures based on Step D, or involving a combination of steps in this region (the data loss was greater for Step D than for Step A and Step B data, which were more advanced at the time of the expulsion). This loss covered entire lottery blocks, affecting treatment and control units alike. While it affects the range of areas to which our results can speak, as well as our statistical power, this loss is not related to the treatment status of units and is thus unlikely to induce bias. Other sources of missing data were the inaccessibility of some regions for safety and security reasons; failures in the field, ranging from loss, damage, or theft of tablets, water damage to paper surveys, or enumerator error in the implementation of surveys or particular questions; survey non-response; and non-response on particular questions by subjects. Figure 5 shows that there is balance between Tuungane and control areas, and between RAPID and survey-only villages, for both village-level and survey-level attrition. K.4 Noncompliance A second threat to validity is that some areas that were selected by lottery to participate in Tuungane did not participate, and vice-versa. Survey data indicates that approximately one in seven chiefs either deny that Tuungane took place in a Tuungane community, or claimed that it did take place when according to records it did not. For all cases with discrepancies between our data and chief reports we asked the IRC to confirm whether the CDR program did or did not take place in these areas. IRC records of where Tuungane did take place matched our records of where Tuungane ought to have taken place in 77% of these ambiguous cases. This suggests that the discrepancy is due either to weak impact, poor recall by chiefs, or enumeration error. The check left 51 cases out of 806 of possible noncompliance and/or database error. For the analysis in this paper we use our database measure of units 55 M Statistical Power Figure 6 presents estimates of minimal detectable effects for our first outcome – share of funds that are not accounted for. This outcome is defined at the village level and so has, ceteris paribus, weaker power than measures defined at the individual level. The vertical axis shows the probability to detect an effect of a given size. The horizontal axis presents possible effect sizes. To construct this figure, we use the real outcome data variance from the control group as well as block structure and conduct simulated analysis given different possible random assignments to treatment and conjectured effect sizes. The figure thus provides the minimum treatment effect beyond which our study design would have an 80% chance to detect the treatment effect as statistically significant. Figure 6: Minimum Detectable Effect Size for Financial Irregularities ● ● ●● ● ● ● 0.00 0.05 0.10 0.15 0.20 0.0 0.2 0.4 0.6 0.8 1.0 Financial Irregularities ATE (Absolute Value) power Notes: This figure presents the power curve. On the y axis, we show the probability to detect an effect of a given size. On the x axes, we present the possible effect sizes. The figure shows the power curve for financial irregularities. To construct this figure, we use the real outcome data variance, and conduct ex-post power analysis. The figure thus provides the minimum treatment effect beyond which our study design would have an 80% chance to detect the treatment effect as statistically significant. 62