On June 6, 2017, Evidence for Policy Design (EPoD) hosted a Global Dialogue event titled “Building Capacity to Act on Evidence: What Does It Take?” Fifty participants – policymakers from around the world, donors, and academics – convened at the Kennedy School for the Dialogue to identify and develop promising strategies and collaborations for strengthening government capacity to use evidence. This event presented insights largely drawn from the four-year Building Capacity to Use Research Evidence (BCURE) program led by EPoD, in collaboration with the Center for Economic Research in Pakistan (CERP) and IFMR LEAD with funding from UK Aid from the UK government.

Presentations at the Global Dialogue tied together the strands of the BCURE program – a major effort implemented in India, Pakistan and other South Asian countries – and formulated broader strategies for applications in a variety of contexts. The purpose of EPoD’S BCURE program is to build the capacity and demand of policy actors to use data and evidence to strengthen systems, better target government resources, and improve decision-making in policy overall. EPoD is doing this through a set of innovative, synergistic activities that aim to identify the constraints that keep individuals from using evidence, overcome those constraints, and build both demand for evidence and capacity for its use. The ultimate goal is to help create a "Culture of Evidence" in the countries of focus.

To accomplish this, EPoD has designed, tested and refined activities in four main areas:

1. Assessment

Early in the BCURE grant period, the assessment team surveyed 1,509 civil servants in Pakistan and 108 in India to identify individual constraints in using evidence. They also conducted an assessment of the barriers to evidence use among civil servants and policy organizations to inform a strategy for training and capacity building. And to understand how these barriers affect policy, they did a qualitative cross-country analysis of data and evidence use by tax departments.
 

2. Pilot projects

Through BCURE, EPoD created opportunities for policymakers to collaborate directly with researchers in building demand and systems to put data or research evidence to use. Each project filled a practical purpose, but also provided the evidence-friendly policymaker with a “quick win” to demonstrate the value of data and research evidence and encourage its use in decision-making. The ultimate goal was to shift organizational practice toward smart use of available evidence.

At the Global Dialogue, EPoD team members Rohini Pande, Charity Troyer Moore, and Eric Dodge described a pilot project they carried out with India’s Ministry of Rural Development (MoRD), the ministry that administers MGNREGA, the world’s largest workfare program, currently serving an estimated 50 million households. Data from MGNREGA was ostensibly “open” and available for monitoring program performance; however, in reality MoRD was storing the data in vast, opaque tables accessible only to very few people who knew how to use them. In the pilot project, Dodge and other researchers worked with MoRD technicians to increase the usability of MGNREGA data through web-based interactive visualizations. This tool immediately saw wide use: over 150,000 hits by over 100,000 users in the year following the launch. The EPoD team has launched a follow-on project – an app for managers called PayDash that flags delays in payments of workfare wages (a common problem in MGNREGA), and identifies exactly which lower-level official is responsible. A randomized trial is currently underway in the state of Madhya Pradesh, where 330 MGNREGA managers have been given the tool and trained to use it. Data from the first months suggest that payments are made more quickly in treatment districts.

Click here to read about all of the BCURE pilot projects.

3. Policy Dialogues

The Global Dialogue, which focused on reflecting on the program itself and developing innovative ideas for capacity-building engagements, was the most recent example of the BCURE dialogues. Earlier Policy Dialogues focused on particular policy issues, brought together the current evidence on them, and sparked research collaborations around them. Topics included environmental regulation in India, and women’s economic empowerment across South Asia.

4. Training

A large portion of the Global Dialogue focused on the fourth BCURE activity, which included the creation of a blended online/classroom training program to improve civil servant’s ability and motivation to use data and research evidence. This strand of the program is poised at a critical juncture: EPoD has built a scalable training platform that includes online modules and customizable classroom teaching materials (read an article on the subject), and integrated the content into mandatory training programs at India’s and Pakistan’s civil service academies. EPoD’s brochure on the training program presents testimonials of participants and case studies on how trainees put their training to use. EPoD is now seeking new partnerships to bring this training program to new audiences, and is also exploring opportunities to deepen learning through more intensive follow up, linkages with on-the-job applications, and organizational or sector-wide training models.

The Global Dialogue highlighted results from BCURE trainings in Pakistan. Sarah Saeed, a director at the National School of Public Policy (NSPP) in Lahore, described the application of the BCURE training modules at Pakistan’s premier training academy to over 1500 civil servants.

Ismail Qureshi, former Rector of NSPP, gave a talk in which he observed that over time, a higher and higher proportion of Pakistan’s civil servants will have received BCURE training. “That is what is going to produce a culture of evidence. When the senior-most officer and junior-most are all on the same wavelength – that is crucial.” He said that this will take another four or five years of engagement. To stop implementing BCURE training before that point would, in his words, represent “a huge loss.” Also, two BCURE trainees – one from Pakistan Railways and the other from Pakistan's Federal Board of Revenue – described how they used their new knowledge to commission surveys and analyze existing administrative data to increase revenues.

In meetings before and after the Global Dialogue, EPoD made steps toward further deployment of BCURE training modules. The Nepal Administrative Staff College will continue to collaborate with EPoD on trainings and other capacity building activities, and plans moved forward to train with potential hosts in Malaysia and Kenya.

5. Hands-on participation

In the afternoon session of the Global Dialogue, participants engaged in a problem-solving workshop based on the framework that guides EPoD’s policy-research collaborations, Smart Policy Design and Implementation (SPDI). Participants, many of them experts in policy and capacity building, addressed questions that have arisen in the course of developing the BCURE training model.

A main takeaway from the Global Dialogue – and of the BCURE program more broadly – was that training on its own can only go so far. Even with demonstrated learning gains (such as 0.29 to 0.56 standard deviations improvement in test scores seen in the BCURE trainings, against a typical benchmark for success of 0.1 SDs), increased capacity on the individual level will only take hold and lead to policy improvements when paired with other capacity-building interventions, such as policy dialogues and pilot projects, and when linked with strategies to overcome some of the institutional and motivational constraints for applying evidence. EPoD is incorporating a number of the insights shared by small groups in the afternoon design session into its strategy for building capacity going forward.