Many of us working in the social sector have designed a ‘perfect’ programme, only to find, at the time of its evaluation, that changes observed were nowhere near what we expected. What happened?
Many things. For one, social change is slow and expecting to, for example, reduce child marriage or violence against women and girls significantly in a two-year period is possibly unrealistic.
Second, since monitoring programme implementation is often not prioritised, we don’t know what the on-the-ground situation is, or whether the programme is being implemented with the fidelity demanded in its design.
But third, and the topic of this article, is our tendency to go from design into implementation without sufficiently testing the waters, without investing enough time in preparatory work and/or without spending enough time supporting community-based facilitators in rolling out the intervention confidently.
‘Good’ programme design isn’t enough
Before moving directly into implementation, it is important to understand the socio-cultural context in which we will implement the intervention, the capacity available at the local level to transact the sessions, the spread of habitations even within project villages and the feasibility of gathering the whole village to programme activities.
Understanding context
In many instances, we develop and implement programmes that aim to effect social change through existing platforms, without clearly understanding whether these platforms are functional and effective. [quote]We develop and implement programmes that aim to effect social change through existing platforms, without clearly understanding whether these platforms are functional and effective.[/quote]I have been engaged in several programmes that taught me that this is not always a reasonable assumption. In one, we aimed to build the capacity of locally elected Panchayati Raj Institution (PRI) members to change attitudes and promote healthy practices among women.
In another, we aimed to work with school management committees (SMCs)—tasked with ensuring enrolment, attendance and overcoming barriers to schooling—to empower communities to ensure regular attendance of teachers and students, and to reduce discontinuation.
In both cases, we had not expected that these leadership platforms would not be strong. In some SMCs, women from the community were not even aware that they were members of the SMCs; and some PRIs were reticent about discussing sensitive issues such as gender roles, child marriage or violence against women.
What was needed was a longer-than-expected preparatory phase, during which we strengthened platforms, raised awareness of and sensitised those in authority about their responsibilities and how to discharge these.
It is important to understand the socio-cultural context in which the intervention will be implemented. (Photo courtesy – Charlotte Anderson)
Understanding available capacity
With an eye on scaling up a successful programme, one always tends to look for local capacity, that is, women (for the most part) from the community who will implement programme activities and sustain them once the programme is over.
In doing so, we identify women who have leadership skills or hold positions of authority in local organisations, orient them about the programme and train them as programme facilitators. Following the one-two week training, the facilitators are expected to mobilise communities and conduct sessions and other activities fairly independently, with supervisory supportive visits from programme implementing teams.
While this sounds like a workable model on paper, there exist many challenges which may inhibit its effective execution. For example, I was engaged in a programme that sought to reduce violence against women by working with women members of self-help groups (SHGs) at the village level. A well-rounded curriculum was developed that aimed to empower women, build their financial literacy and access to savings, transmit new ideas and norms about gender and women’s rights, inform them about available entitlements, and more generally, act as change agents in their communities.
Facilitators were exposed to a participatory training programme,followed by fortnightly SHG meetings that were led by each facilitator, with occasional supervisory support from the implementing partner.
One woman from each group who displayed leadership skills was identified as the group mentor, given training and then expected to conduct some 24 sessions with her group on topics as complex as gender relations, financial literacy, women’s rights and services available to women who experience domestic violence. They were also expected to sensitise other women in their community to become aware of their rights and entitlements.
But many groups didn’t have a single woman who could read and write sufficiently well to be able to convey the essence of each session to the group; most women lacked the skills and self-confidence to speak before a group, even though it was a group of their peers.
This called for mid-course changes: more frequent training programmes and more active participation, rather than just supportive supervision, of project staff.
Related article: Bridging the gap between design and implementation
Understanding geographic spread
In another programme with adolescent girls, groups were formed in each village and a safe space provided for the girls to meet, build peer networks, and gain information and skills. But many villages contained hamlets that were far removed from the main village inhabited by some of the most vulnerable girls (socially excluded castes and tribes, poor, out of school); these girls rarely made the long trek to the safe space and, thus, were effectively excluded from the intervention.
What was needed was a sound understanding of the geography of the village, and the establishment of groups and safe spaces in locations that worked for the most vulnerable. This meant establishing more than one such group in a village, with group facilitators holding sessions in different hamlets on different days of the week or month.
Getting the how right
Give it more time
Even the best evidence-informed programme needs a period of gestation in which contextual lacunae are addressed and delivery capacity built. But we may fail because we don’t recognise this and allow ourselves only a limited time-frame in which to prepare to deliver our dream intervention.
As a result, while we get the ‘what’ right, we don’t get the ‘how’ right.
To understand the ‘how’, we ideally need to pay more attention to on-the-ground realities than we typically do while designing and allocating resources for the programme. A longer preparatory period–perhaps a year, perhaps more–is required to allow us to modify programmes to suit context, build leadership capacity and support community leaders to become effective change makers. [quote]To understand the ‘how’, we ideally need to pay more attention to on-the-ground realities.[/quote]An intensive preparatory phase is not a waste of time; nor is a supportive introductory phase during which project managers play a mentorship role and hone the on-the-ground skills of those tasked with delivering the intervention.
Although we were able to make mid-course changes in all the above examples and although we were able to demonstrate positive effects, I believe that a lengthier preparatory and gestation period may have resulted in a more impressive effect.
Monitor it actively
Often, programme monitoring focuses on the outputs: Did we conduct the number of group sessions we were scheduled to? How many people attended our meeting? Were the topics proposed for this month addressed? [quote]Often, programme monitoring focuses on outputs. Far more rarely do we document process.[/quote]Far more rarely do we document process: What are the perceptions of programme participants–do they find the programme useful? Do they attribute to the programme any change in their attitudes, aspirations or exposure to the outside world? What changes would they advocate? What didn’t they like?
Related article: M&E: Whose job is it anyway?
Likewise, what are the perceptions of ground-level programme implementers: Are project participants receptive to their messages and activities? What are the challenges they face? And finally, observation of project activities is essential.
Together, this kind of documentation of process, together with an assessment of whether activities were carried out faithfully and as planned, will provide deep insight on the kind of mid-course changes warranted, the ways in which activities can be restructured to be more responsive to the preferences of project communities, and the timing and content of refresher training and mentoring support.
Unfortunately, those who have conceptualised and lead the programme may not actively and frequently monitor its progress. The importance of such engagement—observation and process documentation by lead programme implementers—cannot be underplayed.
We are impatient social sector change agents; it is equally important for our donors to understand and accept the need for a longer preparatory phase. Change is slow; starting a programme without adequate preparation and then realising that it isn’t working is too great a risk to take.
