Iqbal Dhaliwal is the global executive director of J-PAL, a global research centre working to reduce poverty by ensuring that policy is informed by scientific evidence. Iqbal has held a number of positions at J-PAL since joining in 2009, and has worked closely with Nobel laureates Esther Duflo and Abhijit Banerjee. He began his career in public service as a member of the Indian Administrative Service (IAS).
In this interview with IDR, Iqbal talks about the institutional partnerships J-PAL has been forming with state governments in India—the barriers they have faced, the approaches that have worked for them, and the learnings they have had.
Photo courtesy: J-PAL
J-PAL has been forming institutional partnerships with various state governments in India to conduct research in line with the government’s priorities. What led to this increased focus on working with governments?
When J-PAL was founded about 16 years ago, we conducted evaluations mostly with small nonprofits. There are obvious benefits to working with smaller organisations—you only have to convince the founders or executive directors, you can make decisions very quickly, and when results come out, it is easier to scale up programmes.
If we want to reach scale, the government cannot, and should not, be bypassed.
But soon we realised that if we want to reach scale, and influence large amounts of development spending, the government cannot, and should not, be bypassed. Therefore, about 11 years ago, we started working with governments. We started participating first on the research side, conducting randomised evaluations. Over time, we moved our focus to see how results from these randomised evaluations could be woven into the policy and programme decisions of the government.
Related article: IDR Explains | Randomised Controlled Trials (RCTs)
What are some barriers that you have faced while working with governments?
When you start working with governments, initially you need to be more opportunistic than strategic. By opportunistic, I mean finding champions in the government who want to work with you, or reach out to you themselves. Essentially, government partnerships tend to be demand-driven in the beginning, when you are trying to get a foot in the door.
The real barriers come when you want to work strategically. For instance, when you want to work with not just departments within a state government, but change the way that the entire government body thinks about evidence, and make evidence a regular part of decision-making.
One of the biggest barriers is the cumbersome rules and regulations you have to deal with when you start working with governments. For example, if you have to sign a Memorandum of Understanding (MoU), you need to figure out which department should sign it, what should it include, and so on. If you were working with a particular official, and they get transferred, it becomes a challenge to ensure that your partnership survives the transfer.
There is also a perception that some governments don’t want their programmes to be evaluated and they won’t appreciate negative results, and there is some truth in it. But we have been happily surprised to find this to be a relatively small issue. There is increasingly more acceptance of rigorous evaluation among government officials—especially when we are able to show them the benefits, like designing programmes with greater impact, and saving money by ending ineffective programmes.
Civil society should try to build close relationships with policymakers so that we can share relevant best practices with the state, and do it at the right time. | Picture courtesy: Pixabay
What are some approaches that have worked for you?
Over the years, we have found templates that work in terms of MoUs. For example, not having a financial component in the original MoU definitely helps. We, at J-PAL, are fortunate that we have an independent revenue stream from our endowment at the Massachusetts Institute of Technology (MIT), and don’t have to depend on government funding.
Another lesson we have learned is maintaining formal paperwork of all our engagements with the government. In the first couple of instances, we would go in and start working without any paperwork other than a couple of email chains here or there. When an official we were dealing with would get transferred, the new one would ask, “Who are you? Why are you here? Show me the paperwork. Show me the government order which authorised you to work here.” So, now we have moved to more formal MoUs and paperwork.
Beyond establishing MoUs, it is important to build a coalition within the government. You could be dealing with, say, just the health department, but you also need to communicate with the finance department, because ultimately, they have to approve the funding. Another key player is the planning department. Many state governments’ planning departments still play a huge role in approving programmes. And then there is the Chief Minister’s office, which often plays a very important coordinating role across these departments.
Another approach that has worked for us is proposing to run evaluations for particular aspects of a programme, rather than the entire programme. For instance, instead of evaluating the entirety of the Swachh Bharat Abhiyan in a state, we could do it on specific behavioural interventions targeted at increasing toilet usage. This still gives us actionable evidence with the potential to improve lives for a large number of people, but it seems less daunting to administrators. This differentiation and prioritisation has been useful to encourage governments to partner with us.
Could you share some of your learnings that can guide civil society organisations looking to partner with governments?
We’ve learned a few important lessons over years of working with governments, especially in India. I’d like to share three of them here.
Let me explain this with an example. The government of Karnataka wanted to install biometric devices in rural primary health centres (PHCs) to monitor the attendance of health workers and doctors. J-PAL was working with the government to evaluate the programme in 150 PHCs in five districts before scaling it state-wide.
The results from this evaluation were quite striking. Initially, the attendance of doctors and nurses started increasing, as was expected. But after 5-6 months, the attendance started falling, and by the time the evaluation was completed, the attendance was lower than it was before the programme. We analysed the results and found that for attendance monitoring systems to have an effect, governments must be prepared to take action against the doctors who are absent. But in the case of Karnataka that was not possible, because there was a severe shortage of doctors. If they took any action against the doctors, the doctors would have just quit and joined a private service.
The government decided to not scale the programme and probably saved crores of rupees. This is important because once a government programme gets started, it hardly ever gets shut down; so, using evidence to not start one is a big step on their part.
Another time, we were working with the Government of Haryana to increase immunisation rates among children in certain pockets of the state. We had to get the word out among the community in a way that minimised the number of children who were left out.
Generally, the first person we think of to spread information in a village is the tehsildar or the panchayat president. But this time, we asked: who is the person in the village who gossips or networks the most? When we posed this question to the people, they did not even list the tehsildar or the panchayat president; instead, they pointed to a grandmother in their community, or someone they went to fetch water with. And so, we identified these people and called them ‘network impactors’, and through them, we started sending out the information about the importance of immunisation.
The results from this low-cost intervention was successful, and the government is now looking to scale this up.
It can be challenging to respond to policy opportunities in real time. Often, civil society organisations may give information to the government when there is no demand for it, and don’t give it when there is actual demand for it. In the process, governments don’t learn from existing knowledge. Civil society should try to build close relationships with policymakers and stay engaged in local politics so that we can share relevant best practices with the state, and do it at the right time.
For instance, if a political candidate says in their election manifesto that they will distribute free laptops, we need to collect the evidence and keep it ready—so the moment they get elected, we can show them studies that say free laptops is not an ideal solution for learning, and so on.
Most people in the government are open-minded towards receiving evidence. The question is how do we give this information and help them understand it?
Similarly, now is the right time for us to be talking about demand side issues around Swachh Bharat. The toilets have been constructed and we need to look at whether they are actually being used or not. So, if we go to the government now, with hard evidence that shows the issues with toilet usage, there is a higher likelihood of them partnering with us.
Surprisingly, we have found that most people in the government are open-minded towards receiving evidence. The question is how do we give this information and help them understand it? Politicians and bureaucrats across the board, even the most corrupt and incompetent among them, want to be able to say, “I ran ‘X’ programme and it had a big impact.” People usually don’t intend to create bad programmes; the problem is that they just don’t know what better options may be out there. And that is where civil society can step in. More often than not, they want evidence—and they do not want an article in the newspaper which says their programme was a massive failure.
Related article: Evidence is about facts, policy is a political decision
How challenging has it been to balance the priorities of J-PAL and those of the governments you want to work with?
I think of it as a Venn diagram. On one side, there are topics that our researchers are interested in studying that governments may not be interested in. On the other, there are programmes which governments want to implement that researchers may not be interested in. We try to find the intersection between the two.
For instance, there are researchers who want to understand, say, the demand curve for clean water or clean air, and the government might not be interested in finding that out.
On the other hand, there are macroeconomic policies like demonetisation, which we do not necessarily expect the government to want to evaluate through a randomised evaluation, in part because it is not realistic to randomise some states or districts to participate in demonetisation and others to remain in a comparison group. So, perhaps we would not evaluate that either.
Then there are policies such as national health insurance schemes (e.g. Ayushman Bharat), where both central and state governments would want to know what is the most efficient way to give health insurance to tens of millions of people, what the impact is going to be on the government hospitals, how to improve monitoring, and so on. These are areas where an RCT would be most pertinent.
Even within programmes where an evaluation would add value, it is important to determine which component should get evaluated. For instance, for a programme that administers HPV vaccines, there are clinical studies that already show that it reduces cervical cancer in girls. What we need a randomised evaluation for, is to determine how best to incentivise young women and their families to take these vaccines.
Our experience partnering with governments in India and around the world has shown us that there is great potential for randomised evaluations to have a huge impact on improving people’s lives. We need more partners—not only governments, but also local researchers, civil society groups, and visionary funders—to help advance this movement.
—
Know more
- Explore results from more than 1000 evaluations conducted by J-PAL in 85 countries.
- Read about J-PAL’s institutional partnerships with various state governments such as Odisha, Haryana, and Andhra Pradesh.
- Read more about the pathways through which evidence can have an impact on policy and case studies that illustrate partnerships leading to policy impact.
- Watch this TED Talk by Esther Duflo, co-founder of J-PAL, on the value of testing solutions through randomised trials to alleviate poverty.
Do more
- Enrol in the online courses Challenges of Global Poverty (free) or Data, Economics, and Development Policy, to understand how to tackle development challenges from an evidence-based perspective.
