Despite recent growing demand from funders and governments, rigorous impact evaluations in Latin America and the Caribbean remain the exception rather than the rule. Many commissioned impact evaluations are methodologically weak, and thus only marginally useful in assessing the impact of social interventions. Other impact evaluations feature strong research methodologies at their conception, but face considerable institutional challenges during key points in the design and implementation phases. This paper identifies some of the barriers that limit the design and implementation of rigorous impact evaluations in this region, as well as several enablers to the successful design and implementation of such evaluations. The paper also outlines some key practices for designing and implementing high-quality impact evaluations in Latin America and the Caribbean. We use a case study methodology that  combines our experience designing and implementing impact evaluations in three ongoing or recent social programs in El Salvador, Jamaica, and Mexico.
Close

In recent years, the international community has expanded efforts in program evaluation to improve the accountability of development projects. This paper presents approaches to implementing state of the art evaluations in rural electrification projects, taking into account specific challenges that researchers face in such interventions. It suggests an approach to assess impacts before an intervention is implemented by surveying the yet non-electrified target region of the project and, in addition, an already electrified region. Besides delivering robust evidence on impacts, results from such ex-ante evaluations deliver insights for the project design, thereby reducing the gap between evaluation researchers and practitioners.
Close

This research paper intends to quantify targeting performance in terms of the efficiency of the
National Plan for Social Emergency Assistance (PANES) implemented in Uruguay between 2005 and 2007 and determine its impact on relevant issues such as school attendance, child labor and the labor market. For this analysis, we used 2006 and 2007 Continual Household Survey (ECH) data.
Our outcomes show that in 2007, 27% of the eligible households in Montevideo and other urban areas were not able to enter the program. In addition, less than 2% of the non-eligible households received program benefits. Even though this allows us to conclude that program targeting was actually not high, targeting indicators show noticeable improvement when compared to indicators for 2006. Additionally, there are no signs of discontinuity around the thresholds set in different regions to gain access to the program. This invalidates the regression discontinuity method used to evaluate program impact.
As regards the evaluation, estimates made using the propensity score matching estimator show that PANES has not had significant impact on school attendance or child labor. In addition, adverse labor market effects are observed in terms of worked hours in urban areas other than Montevideo for both men and women. Quantitatively, a reduction in the number of worked hours by individuals who participated in PANES is observed, around 1.4% for men and 8% for women. It is worth mentioning that effects observed on the labor market relate only to worked hours and not to participation in the labor force. As regards informality, a non significant increase is observed. No impacts on the labor supply, labor force participation or informality are observed for Montevideo.
The absence of quantitatively significant effects is not surprising since PANES can be described as a hybrid between food and basic need programs and human capital development programs. Even though the act under which PANES was created establishes eligibility requirements, there is actually no information available on the level of compliance with program requirements (Amarante, Burdín and Vigorito 2008).
Finally, our conclusion is that despite the fact that the purpose of PANES was to solve serious problems faced by Uruguayan society, such as indigence, these were only partially solved because PANES seeks a short term solution to a multiplicity of issues such as employment, school attendance, health, etc, addressing them as a whole and solving these issues as a whole is difficult. Although regional conditional transfer policies address several issues as a whole, they focus on only one of them, usually “the accumulation of human capital”, which is considered essential to eliminate inter-generational poverty.
Close

This paper presents estimates of the impact of programs for unemployed workers on the performance of program beneficiaries in Mexico. We emphasize the significance of applying methodologies capable of avoiding statistical bias attributable to unobserved variables when measuring the impact on earnings and allowing to us properly estimate unemployment duration and work status after exiting from unemployment.
Using a technique that combines the matching method with the double difference method (pre and post-intervention differences in differences between the treatment and control group) we measure the impact of SICAT on unemployed males with previous working experience who participated in the 2002, 2003 and 2004 programs. Outcomes using this method do not support the hypothesis that SICAT has a positive impact on the earnings per hour of program participants three months after finishing the training provided by the program. Furthermore, when the impact is statistically significant, such as in years prior to 2004, the effect was the opposite to the one that was expected, as if those participating in SICAT had been stigmatized in the labor market to the extent that it turns out to be counterproductive.
For the SAEBE program, we measured whether the probability of finding employment in the formal sector increases for program beneficiaries and whether they exit from unemployment faster when, in addition to financial assistance, they are provided employment services.
The purpose of these estimates is to identify the challenges future evaluations will have to face to be able to measure the impact of the SAEBE and SICAT programs on beneficiaries in a more adequate and accurate fashion. Among them, we emphasize the need to change the way surveys are conducted and control groups identified, improving data collection of job statuses trajectories, and the need to incorporate general equilibrium conditions in the labor market when measuring the impact of programs that cover a significant number of unemployed workers.
Close