EPoD's 2021 in Review

EPoD's 2021 in Review

By: Nathalia Bustamante, Samia Sekkarie

The importance of evidence for policymaking has been underscored by the COVID-19 pandemic. 2021 demonstrated how available evidence can be used to design policy responses to unpredictable and unprecedented situations: Data is not only a powerful tool in navigating the trade-offs of policy decisions; it is also a resource for building trust and common understanding.

EPoD’s vision has become more relevant than ever: “A world where evidence drives continuous improvement in states, systems, and societies for better lives.”  

This continuous improvement can only be achieved through collaborations between researchers, individuals and institutions working on the ground, in different countries and contexts. In 2021 we were grateful more than ever for our community of academics, staff and policy counterparts around the globe. Working directly with those who implement policy necessitates the adaptation and continuous improvement of our research to impact the lives' of the most vulnerable.  In 2022 and beyond, it is this teamwork that makes us all stronger and able to tackle the world’s most complex problems. 

EPoD Staff

EPoD staff, fellows and faculty gathered virtually in the Spring.

 

As we look back at 2021, it is important to stop and celebrate the progress we have made, both in expanding the evidence base of what works through research, and in empowering policymakers and practitioners to produce and use evidence through capacity building. EPoD’s mission was pursued through rigorous, on-the-ground embedded research on the pressing challenges of our incipient decade. Additionally, we expanded and updated our evidence-driven capacity building programs to better prepare leaders around the world to use evidence to tackle their own challenges. Aside from strengthening this culture of evidence based policymaking, the expansion of our programs increased our accumulated knowledge and helped make our trainings more adaptable and relevant than ever.  

This overview of 2021 highlights our impact, research, trainings, media mentions and events, both recognizing our achievements and inspiring much of the work we have in front of us in 2022.  

Research 

EPoD’s research unfolds into different areas of research and practice – reflecting the diversity of backgrounds and expertise of our scholars. All of them share a commitment to working with policymakers and practitioners in the field, aiming to offer concrete steps to improving policy, grounded on large-scale field experiments and cutting-edge analytical methods through our Smart Policy Design and Implementation (SPDI) approach. 

 

Education

  • Busting the myth that private schools are only for the elite in Pakistan. Our Learning and Educational Achievements in Pakistan Schools (LEAPS) research on Pakistan’s education sector found that 89% of the private schools in the country are extremely low-cost, catering to the bottom of the pyramid. Yet, they are on average outperforming public schools by significant margins. 

  • Bride and prejudice: the price of education. Bride price, or the custom by which the groom makes a payment to the family of the bride, may influence girls’ level of education attainment. Research co-authored by Nathan Nunn identified that bride price increases with education. Hence, girls from bride price ethnic groups were more likely to be enrolled in school. The findings suggest that policymakers must consider how proposed policies interact with cultural traditions.  

  • Changing gambling behavior through experiential learning. How can adult learning be influenced by their own experiences? In a randomized field experience in South Africa, a simple dice rolling game was used to inform participants about the odds of winning the national lottery. The research, co-authored by Shawn Cole, showed that individuals who need many attempts to roll two sixes play the lottery significantly less than the control group, while those who need fewer attempts adopt the opposite behavior. These findings provide practical guidance for correcting biases in beliefs through interventions that promote brief and real experiences.  

Environment and Energy

Health

Labor

  • Diversity and team performance in a kenyan organization. An RCT conducted by Vincent Pons and colleagues in Kenya demonstrates that horizontal ethnic diversity (between teammates) decreases team performance, while vertical diversity (between teammates and their supervisor) often improves performance.  Findings suggest that horizontally homogeneous teams organized tasks in a more efficient way, while vertically homogeneous teams exerted lower effort.  

  • The psychosocial value of employment. In settings where employment opportunities are scarce, the inability to work may generate psychosocial harm. A study by Reshmaan Hussam et al. measures the impact of employment on mental health among refugees, and shows that the benefits go far beyond those of income alone.  

  • Information, loss framing, and spillovers in pay-for-performance contracts. This field experiment conducted by Sebastian Bauhoff et al. in primary health facilities in Nigeria assessed the impact of incentives and information on workers’ performance. Results suggest that a small incentive captures most of the impact, implying that pay-for-performance contracts can be made significantly more cost-effective. 

  • Immigration and occupational comparative advantage. Gordon Hanson and Chen Liu found that the job choice of highly skilled foreign-born workers in the US correlates strongly with their country of origin. In the paper, they evaluate the causes of immigrant sorting across jobs.  

  • Search and information frictions on global e-commerce platforms: evidence from AliExpress. Research by Jie Bai and co-authors shows that the mass of firms operating on e-commerce platforms can congest consumer search, and hinder market allocation towards better firms. The consequence is that initial demand shocks (‘luck’) can determine a firms’ growth and market allocation, rather than productivity or product quality. 

  • On the persistence of the China shock. The China trade shock plateaued in 2010, and this study by Gordon Hanson et al. evaluates its impact on a wide range of outcomes over the period 2000 to 2019. Findings indicate that import competition from China induced changes in income per capita across local labor markets. The effect holds even when estimating consumer benefits of rising trade. 

  • Targeting high ability entrepreneurs using community information: mechanism design in the field. Identifying high-growth microentrepreneurs in low-income countries remains a challenge, due to a scarcity of verifiable information. A cash grant experiment in India conducted by Reshmaan Hussam, Natalia Rigol, and Benjamin N. Roth, demonstrates that community knowledge can help target high-growth microentrepreneurs. However, there is evidence that community members distort their predictions when they can influence the distribution of resources.  

Social Protection

  • Employment structure and the rise of the modern tax system. Using evidence from a new micro-database, Anders Jensen establishes three stylized facts explaining how intimately connected the expansion of the income tax base is to a country’s development. 

  • Tax administration vs. Tax rates: evidence from corporate taxation in Indonesia. Which has better outcomes for developing countries—improving tax administration or increasing tax rates? Based on reforms of corporate taxation in Indonesia, a study by Rema Hanna, Post-Doctoral Fellow Mayara Felix and colleagues shows that improved tax administration is equivalent to raising top rates by 8 percentage points. Ultimately, the findings suggest that improved tax administration can have significant returns for developing countries.  Watch a video about the research below. 

 

 

  • Food vs. Food stamps: evidence from an at-scale experiment in Indonesia  Rema Hanna co-authored research that sought to identify the difference in impact between welfare programs that provide food vs. programs that provide food-vouchers. Their analysis was based on the Indonesian government’s randomly phased transition from in-kind delivery of subsidized rice to equivalent vouchers. The results indicate that vouchers led to greater impacts on poverty, whilst substantially decreasing the administrative costs of benefits delivery.  

  • The economic consequences of increasing sleep among the urban poor. A study from Gautam Rao o et al. concluded that short afternoon naps at the workplace improve outcomes in low-income urban workers, with significant increases in productivity, psychological well-being, and cognition, but a decrease in work time.  

  • Encouragement and distortionary effects of conditional cash transfers. While conditional cash transfer (CCT) programs are designed to encourage some desirable behavior, an unintended consequence may be to distort recipients’ actions in ways that lower social welfare. Post-Doctoral Fellow Joeri Smits and co-authors designed an experiment to study the role of the transfer size in shaping distortion, and implemented it in a cash transfer program conditional on seasonal labor migration in rural Indonesia. They found that when the transfer size exceeds the amount required for travel, distortionary effects dominate and migration earnings decrease.

Financial Inclusion

  • Changes in social network structure in response to exposure to formal credit markets. Research co-authored by Emily Breza with Abhijit Banerjee, Esther Duflo et al found that formal financial institutions can have far-reaching and long-lasting impacts on informal lending and information networks. Studies in villages in Karnataka and Hyderabad, in India, discovered that exposure to credit access was associated with subsequent fewer social relationships between households.  

  • After the burning: the economic effects of the 1921 Tulsa Race Massacre. The 1921 Tulsa Race Massacre resulted in the looting, burning, and leveling of a once-thriving Black neighborhood. Research co-authored by Nathan Nunn showed that it not only led to economic losses, but also to declines in home ownership and occupational status – especially in areas with more newspaper coverage of the massacre or in communities that, like Tulsa, had high levels of racial segregation.  

  • Borrower leakage from costly screening: evidence from SME lending in Peru. Commercial lenders in Peru suffer leakages in their loan approval process. Research by Ben Roth et al. highlights that information spillovers between lenders may operate outside of credit registries.  

  • Learning to navigate a new financial technology: evidence from Bangladesh. How do consumers learn to use new financial technology? As more people access mobile money, there has been concern that financial middlemen can exploit mobile money customers. Emily Breza et al. find evidence from Bangladesh that experience helps consumers navigate this new financial technology, shedding light on where traditional financial training and education interventions would have the highest marginal effect.   

Governance & Accountability

  • The political economy of state employment and social unrest in China. State employment is used to pacify social unrest in China. Jaya Wen uses variation in a conflict between Uyghur separatists and the government to establish that, in years and counties with a higher threat of unrest spillover, state-owned firms hire more male minorities (the demographic most likely to participate in ethnic conflict). The research also found that state employment increases after natural disasters and poor trade shocks, indicating that public jobs play a more general stabilizing role as well.  

  • Public disclosure as a political incentive: evidence from municipal elections in India. Insights from a study co-authored by Michael Walton suggest a positive relationship between public disclosure of performance and investments, and between investments and voter support. Analyzing council election in urban Delhi, India, the authors found that councilors who anticipated that performance reports would be published before an election invested more in infrastructure, with positive impacts on re-election.  

  • Voter mobilisation and trust in electoral institutions: evidence from Kenya. Text messages that intended to mobilize voters in Kenya were successful in increasing electoral participation. However, they also decreased trust in electoral institutions after the election - especially for individuals on the losing side and in areas that experienced election-related violence. The large-scale experiment conducted with Kenya’s Electoral Commission by Vincent Pons and colleagues raised potential mechanisms that could account for the intervention’s unexpected effects. 

Training 

With the aim of increasing policymakers' capacity to navigate evidence, EPoD designed, piloted and updated several training programs for policymakers and practitioners. We transformed many of our workshops to be taught online, while maintaining the same levels of engagement, learning, and enthusiasm as prior years. In a context of great need and ease of connecting with worldwide audiences, we hosted 150 participants in our HKS Executive Education programs, and our collaborators trained 1064 participants in the Building Capacity to Use Research Evidence (BCURE) program. Together, those programs reached professionals from 51 countries, contributing to our goal of equipping students, policymakers and practitioners worldwide with the skills to incorporate evidence into policy design and implementation.  

New Programs with International Organizations 

  • New executive course on policy responses to COVID shock, in partnership with the African Development Bank. EPoD collaborated with the African Development Bank to train the Bank’s Country Economists on effective post-COVID-19 responses and recovery strategies for Africa. The new course focused on a multidimensional curriculum that included fiscal and macro policy, trade, tax and resource mobilization, labor markets, and social protection systems – lending on EPoD’s Smart Policy Design and Implementation (SPDI) framework. Between January and February 2021, seventy economists from the Bank participated in the interactive program and have since contributed to guiding member countries on their fiscal and social policy responses to the crisis. Program faculty included Rema Hanna (program chair), Matt Andrews, Hakeem Belo-Osagie, Bruno Crépon, Karen Dynan, Anders Jensen, Asim Khwaja, Rob Lawrence, and Dani Rodrik.  

  • In collaboration with the World Bank, EPoD conducted a training for Sri Lankan policymakers on the use of RCTs for program and policy evaluation. The goal was to build a common understanding around the goals and methods of the kind of research EPoD does, focusing on why and how RCTs are useful in evidence creation. Professors Rema Hanna and Ben Olken led the training, with instruction by EPoD staff members Alexa Weiss and Charlotte Tuminelli. 

Data-driven trainings  

  • Our BCURE training program is aimed at empowering policymakers and civil servants with the knowledge and motivation to generate, use, and communicate data and evidence for more effective policymaking. The BCURE program is centered around our Training of Trainers (ToT) model – in which EPoD trains civil service academies and universities to incorporate BCURE content into their syllabi. Our 2021 efforts on this front included continued partnerships for scaling trainings in Pakistan, India, and Morocco. 

    • Civil servants in Pakistan and India were trained using EPoD’s curriculum, through continued collaboration with CERP and Inclusion Economics, respectively. 

    • Through the Morocco Employment Lab, EPoD hosted a series of workshops for over 50 instructors from Moroccan government agencies and universities. The rollout of this program was more than an expansion to another country – it was also the first time the BCURE curriculum was delivered completely in French. The final recipients of training were both graduate students and government officials from a range of different ministries. 

Leading Smart Policy Design Executive Education Program  

In 2021, our Leading Smart Policy Design Executive Education program prepared 75 leaders to design and implement policies that address social and economic problems around the world. Using EPoD’s Leading Smart Policy Design framework, the program provided practical frameworks for analysis and hands-on experience using a smart, systematic, and collaborative approach to theory- and data- driven innovation. This year’s final presentations covered digital financial tools for the poor in Nigeria, education quality during crises in Lebanon, and addressing US racial wealth disparities at the local level. Read more >> 

Policy Impacts 

Our goal throughout our activities is to inform evidence-based policymaking in order to promote more effective and efficient policies and programming. This year, we have several stories of impact that demonstrate the diverse pathways through which our research and training “drives continuous improvement in states, systems, and societies for better lives.” 

Technology and Local State Capacity in Ghana

EPoD affiliate Anders Jensen and co-authors conducted research analyzing the association between technology use and property tax billing, collection and enforcement by local governments in Ghana. The project randomized the use of a new revenue collection technology and found that investments in technology alleviate information constraints and improve the progressivity of the tax system (making it more likely that people with high income-assets pay property taxes), thereby increasing tax collection. The team presented their findings to the Ministry of Finance and to the Vice-President’s Office, who has since officially encouraged all local governments to adopt technology for property tax collection purposes. Additionally, the local institution with whom the researchers were collaborating was invited to join the effort of setting up an electronic procurement platform that would enable scaling technology to the country’s 256 local governments. 

Enhancing Education Quality in Pakistan

The LEAPS team launched a large-scale post-COVID Targeted Instruction Intervention in 1,500 public schools in Pakistan, in partnership with the Federal Ministry of Education and the Khyber Pakhtunkhwa (KP) Provincial Education Department. The program is designed to address student learning gaps, exacerbated by COVID-19 induced school closures, and help students catch up grade levels by building foundational skills in language and math. The program introduces a remedial curriculum and associated pedagogy targeted at students’ current learning levels, implemented by existing public school teachers with the help of an easy-to-use technology tool. The technology is designed to take the administrative burden of targeted instruction off teachers, automating processes like sorting children according to learning levels, analyzing test score data, and recommending what to teach next.   

The Targeted Instruction Program (TIP) was piloted with 16 NGO-run schools serving approximately 3,000 children, before being taken to 1,250 public schools in KP and around 250 federally administered public schools, impacting over 300,000 children. Thus far, over 6,000 public school teachers, head teachers and education officials have been trained in how to implement the program, including the curriculum, pedagogy, and technology use. If successful, the government has expressed interest in scaling up the program to over 32,000 schools in KP with a projected impact on over 4 million children, as well as to other provinces in the country. 

This is a significant achievement: as policymakers searched for viable responses to an education emergency brought on by the global pandemic, LEAPS used intensive policy engagement and relationship-building to seize the opportunity created by this favorable policy environment and launch TIP. While the team anticipates that the results of this intervention will inform further policy action in Pakistan, they also hope their findings will be relevant and insightful for the entire global education community.  

Rebuilding the Social Compact: Understanding Service Delivery and Property Taxes in Pakistan 

Asim Khwaja and co-PIs are collaborating with the Government of Punjab to evaluate a series of reforms that enables taxpayers to better and more credibly express preferences for local urban services and strengthens the link between these services with the taxes they pay. The objective is to improve tax collection rates and strengthen citizen faith in the government’s ability to deliver services. The impact evaluation has led to the expanded role of employees at the Excise, Taxation & Narcotics Control (ET&NC) Department of the Punjab Government from tax collectors to liaisons between citizens and the state. Beyond simply collecting taxes, they now elicit taxpayer preferences for services and inform them about the tax-service link to encourage voluntary tax compliance. This has generated considerable excitement in the government. These reforms offer the possibility of reframing a conventionally transactional engagement into a long-term, sustainable relationship based on mutual respect and voluntary compliance. 

Evaluating Social Protection Programming in Indonesia

J-PAL Southeast Asia and EPoD are evaluating the program effects on labor, consumption smoothing, and financial behaviors of a vocational training and cash transfer program to inform future program delivery.  The program, Kartu Prakerja, is a novel governmental program run by Manajemen Pelaksana Program Kartu Prakerja (PMO) that combines vocational training with cash transfers to develop work skills, promote employment, sustain families, and promote digital financial inclusion. Launched in early 2020, it is a key component of Indonesia’s social protection policy, with millions already enrolled. In order to help inform program implementation in 2022, the team presented their preliminary findings of the impact evaluation to government officials and outside researchers, and then on a public webinar hosted by the PMO. The executive summary of the research findings is also posted on the PMO’s website.  

 

Informing Transition from Food Delivery to Food Vouchers in Indonesia  

J-PAL Southeast Asia and EPoD researchers evaluated the transition of the largest social assistance program in Indonesia providing subsidized food to the poor, from in-kind delivery of rice to electronic food voucher distribution, studying program effectiveness and also relative delivery costs. After conducting a randomized impact evaluation, the results were presented to the Government of Indonesia and at a public webinar hosted by J-PAL SEA. Throughout 2021, the Indonesia’s e-voucher program for food assistance formed a key part of Indonesia’s social safety net during the COVID-19 pandemic. 

Rebuilding the Social Compact: Understanding Service Delivery and Property Taxes in Pakistan 

Asim Khwaja and co-PIs are collaborating with the Government of Punjab to evaluate a series of reforms that enables taxpayers to better and more credibly express preferences for local urban services and strengthens the link between these services with the taxes they pay. The goal is to improve tax collection rates and strengthen citizen faith in the government’s ability to deliver services. The impact evaluation has led to the expanded role of employees at the Excise, Taxation & Narcotics Control (ET&NC) Department of the Punjab Government from tax collectors to liaisons between citizens and the state. Beyond simply collecting taxes, they now elicit taxpayer preferences for services and inform them about the tax-service link to encourage voluntary tax compliance. This has generated considerable excitement in the government. These reforms offer the possibility of reframing a conventionally transactional engagement into a long-term, sustainable relationship based on mutual respect and voluntary compliance. 

Media Coverage 

The impact of EPoD’s research is potentialized by the circulation of our work in different media outlets. EPoD affiliates were featured in several national media outlets, such as The New York Times and The Economist; in many in-country media outlets such as Dawn.com in Pakistan, as well as in cross-national vehicles, such as the World Bank Blog. Finally, Harvard also featured our work in the Scholars Program. Below, you will find some of the media coverage of EPoD’s associated research and activities:  

National Media Coverage

National Media Coverage 

  • Can trade work for workers? Gordon Hanson explored how trade policy can and cannot be used to help workers hurt by globalization in this article for Foreign Affairs. He advocates that rather than adopting protectionist measures, the U.S. Government should focus on providing robust direct assistance and creating a stronger safety net for future generations of workers. 

  • How American leaders failed to help workers survive the 'China Shock'. Gordon Hanson’s latest study on the China Shock was covered by NPR’s Planet Money. His research suggests that even though China reached the limits of its model of economic growth in 2010, the impacts its expansion had on American communities that relied heavily on manufacturing persists.  

  • Development economists get training at the intersection of research and practice. EPoD’s Research Fellows Program was highlighted by Harvard Kennedy School’s website. The program, that already mentored over 120 research fellows since 2008, brings early-career development economists to Harvard Kennedy School to work with faculty on research engagements. 

  • HKS faculty reimagine the world of work. Morocco Employment Lab’s research and capacity building efforts were featured among significant initiatives of HKS Faculty and Researchers on shaping the future of the workforce.  

  • Incentivizing higher-quality agricultural outputs. Jie Bai’s research about Asymmetric Information, Consumer Learning and Seller Reputation was cited on J-PAL's article on opportunities for higher-quality agricultural outputs in low- and middle-income countries. 

International Media Coverage

  • How has Covid-19 affected children and teachers in non-state schools in Pakistan? The Oxford Policy Management Podcast interviewed LEAPS’ Director Zainab Qureshi about the implications of the pandemic for non-state schools, students and teachers in Pakistan. Low-fee private schools are a significant source of employment for women, and with school closures their salaries and employment are at risk.  

 

  • COVID-19 & low-fee private schools in Pakistan: have schools closed, girls dropped out, and women lost jobs? LEAPS' Director Zainab Qureshi was interviewed for Global Schools Forum on the impact of COVID on low-fee private schools in the country. Initial data suggests that COVID-19 related school dropout rates in Pakistan may be higher for boys, who were more easily absorbed into the workforce.  

  • Holding up half the sky. Data on Pakistani school enrollment and student achievement from the LEAPS was cited in this Op-Ed by Dr. Ayesha Razzaque for the Pakistani newspaper The News. Using data collected by LEAPS, Dr. Ayesha Razzaque debunks the myth that private schools in Pakistan are only accessible to the elite. 

Events 

EPoD’s affiliates participated in several events in 2021, sharing their research findings and expertise. Below, a few of our hand-picked favorite participations, available to replay: 

Digital Events & Webinars

  • EPoD affiliates participated in several of Harvard Kennedy School’s Faculty Webinar Series. On “Using Data to Create Effective Policy in Uncertain Times”, EPoD Faculty Director Rema Hanna moderated a panel featuring Matthew Andrews, Asim I. Khwaja, and Karen Dynan on how to use data effectively in making policy decisions, especially in the context of the COVID-19 recovery.  

 

 

 

 

 

  • The Morocco Employment Lab hosted a webinar in which Prof. Bruno Crépon discussed active labor market policies, its potential and limitations, in an economy with low job creation.