Do you ever look around certain rooms, most rooms, and find yourself dismayed at the (lack of) representativeness of it? Some exclusions or under-representations are visible at first glance, others take much longer to be identified. And yet, they exist.
In India, this is particularly salient in the gender composition of enumerator teams. At IDinsight and DataDelta, we have consistently observed a low representation of female enumerators in our teams in India. Despite recognising the importance of hiring more female enumerators for reasons of equity, data representation, and data quality we struggle to do so.
Why is this the case? In part, it’s because we receive fewer applications from women for enumerator roles than their male counterparts. This is consistent with India’s female labor force participation, and partly a result of social norms and other barriers that limit employment opportunities for women. Through our conversations with enumerators, we realized that some unknown biases might also permeate our hiring process and hinder women’s ability to take up these positions. In order to investigate this, we analyzed our hiring data across three DataDelta projects using a gender lens. We studied each step of the process to determine if any of our hiring considerations systematically disadvantage female candidates and developed recommendations to address them.
We found that the only stage at which female candidates score lower than their male counterparts is one that accounts for their access to assets and prior work and educational opportunities. These included access to a bank account, motorbike, driving license, post-graduate degree, and work experience. In interviews, training sessions, and field pilots, women perform at least as well as men. Relaxing these criteria has helped make our hiring process more inclusive and our teams more representative.
We also wanted to know if lowering these criteria had any implications for the productivity or quality of data collected by our enumerators. We will be exploring this, and the relationship between enumerator gender and data quality, in our next blog post. In this blog, we share our process for identifying barriers to employment opportunities for enumerator roles in DataDelta. We hope that others might use similar approaches to identify barriers to employment opportunities at their organizations, and as a sector, we can move towards more inclusive hiring practices.
How we approached understanding the barriers to employment
For each project, DataDelta aims to hire a team of hyper-local enumerators who are demographically representative so they will be familiar with the local language and context. This improves the degree of comfort between the enumerators and respondents. It also mitigates the costs of arranging transport and accommodation for surveyors for the period of data collection.
i) Round I: Application form
DataDelta’s hiring process for enumerators comprises three stages – a review of the candidate’s application form, an interview, and their performance in training and field pilot sessions. The application form typically includes questions about the candidate’s name, age, gender, and home district; language familiarity; educational qualifications; (relevant) work experience; and access to a motorcycle, driving license, Permanent Account Number (PAN), and a Bank Account. At the first stage, each of these criteria is a prerequisite for being shortlisted for the subsequent round i.e. the interview. This shortlisting of candidates from their form responses takes place through an automated process.
In the first project we analyzed, our team followed the traditional DataDelta model of surveyor hiring. We observe that for every 100 male applicants, ~38 were called for an interview; for every 100 female candidates, only ~13 made it through – a 25 percentage point difference. Keeping in mind women’s mobility, education, and labor force participation rates in the country, especially in rural areas, such stark differences in men’s and women’s access to motorbikes, driving licenses, postgraduate degrees, and prior work experience is not surprising.
In our second project, the team had relaxed the requirements for motorbikes and driving licenses for all applicants, and instead relied on public transportation systems and rented auto-rickshaws for enumerator’s commute. Based on the auto-shortlist, ~30% of the male applicants and ~24% of the female applicants were selected – a 6 percentage point difference. We observe that relaxing the requirements for motorcycles and driving licenses alone significantly bridges the gap between the percentage of male and female applicants shortlisted at this stage.
In an attempt to recruit an enumerator team with equal gender representation, the third project team had also relaxed considerations of post-graduate degree and prior work experience for all applicants. Instead, the team provided rented cars, extended training sessions, and increased field pilots. Post the auto-shortlist on the basis of home district, language familiarity, undergraduate degree, and access to PAN and Bank Account, 48% of male and 45% of female applicants were selected for the subsequent round – a significantly improved 3 percentage point difference.
We observe that the list of access-related prerequisites at the first stage of the enumerator hiring process systematically disadvantaged female candidates compared to their male counterparts. When we relaxed these requirements, the gap between the proportion of male and female candidates selected for the subsequent stages was reduced. Their performance in these rounds is detailed below.
ii) Round II and III: Interview and training and field pilot sessions
As per the process, those shortlisted were interviewed and assessed virtually by the field leadership team with questions on math, logical reasoning, and case studies to evaluate their aptitude for enumeration. While technical questions were included in the first two projects, they were replaced by more general acumen-related questions for the third project due to more flexible work experience requirements. As shown in the graphic, across all three projects, irrespective of the variation in hiring considerations, female candidates performed slightly better than their male counterparts and a higher proportion of them were invited for the training round. This is substantial as it shows that despite lower educational qualifications and lack of prior work experience, female candidates have at least as much an aptitude for enumeration, in their interviews, as their (on average, more educated and experienced) male counterparts.
As the last step of the selection process, shortlisted candidates were invited to participate in training and field pilot sessions. The sessions took place in a centralized location and enumerators are remunerated for their time. Here, candidates are trained on the objectives of the survey, the survey instrument, DataDelta’s data collection protocols, and data quality checks. Post the classroom sessions, candidates hone their skills during field pilots. The lead evaluates candidates based on their attendance, participation, daily quiz scores, and performance during field pilots. Those shortlisted form the final enumeration team. A similar trend emerged across all three projects: a higher percentage of female candidates were selected for the respective enumeration teams based on their performance. This highlights their commitment, initiative, and ability to grasp the work despite having, in most cases, less education or work experience than their male counterparts.
We observed that the interview, training, and field sessions did not present any systemic disadvantage to female candidates. In fact, despite the differences in the educational qualification and work experience of the two gendered sub-groups, on average, female candidates performed at least as well as their male counterparts in these stages of the hiring process.
iii) Final team compositions and data quality
While such relaxations have played, and will continue to play, a crucial role in hiring more representative enumeration teams; renting private vehicles, and extending the number of days for training and pilot sessions comes with budgetary implications. Beyond principle-based rationales of fairness, equity, and inclusivity; how do we justify these additional costs to our partners? We analyzed productivity and data quality indicators for our enumerators disaggregated by gender. While we will share our detailed findings in our next blog, we found that relaxing the hiring criteria had no negative implications for the productivity or quality of data collected by our female enumerators.
Key takeaways
Analysis of our hiring data through a gender lens has cautioned us against both the risks and the futility of instituting considerations of access as entry barriers for employment opportunities.
1. First, the risk: Our analysis shows that women perform less well than men at only one point in our hiring process – the point of access. That is when they face the prerequisites of access to motorbikes, licenses, graduate degrees, and work experience at the application stage. In a hierarchical society like ours, access is largely a function of various axes of privilege. While we have only been able to analyze our hiring data from a gender lens so far, it is easy to imagine that the considerations of access that bias our process against female enumerators would also risk biasing it against other historically marginalized groups. By instituting them as entry barriers for employment opportunities, we risk perpetuating the very same social inequalities that we seek to eradicate. On the corollary, relaxing such barriers has helped make our recruitment process more fair.
2. Second, the futility: We also observe that even when we lower requirements for educational qualifications and prior work experience when given the opportunity and training, female candidates and enumerators consistently perform at par with their male counterparts. This includes their performance in interviews, training, field pilot sessions, and actual data collection. This goes to show such qualifications are inadequate indicators of talent, and the skills required for enumeration are highly teachable. Instituting practices that allow candidates who showcase the underlying acumen for enumeration to be recruited and trained, rather than restricting hiring to those who already have prior work experience, has helped us make our hiring processes more inclusive.
This experience has taught us that even when operating in deeply unequal societies, it is possible to structure hiring, training, and performance processes to be equitable, and to uncover and nurture acumen and potential in female candidates that barriers to access otherwise obscure. We intend to use these findings to make our enumerator hiring process more inclusive by de-linking considerations of access with opportunity, to the extent possible. While it may not always be possible to hire 50% female enumerators, we do commit to more meaningfully fulfilling our role as equal-opportunity recruiters by reducing barriers to entry for female enumerators, finding cost-effective ways to support their learning and performance, and working with partners to understand the equity and impact benefits of doing so.
Acknowledgements
With thanks to the DataDelta Gender & Intersectionality Working Group – Sarah T. Lucas, Krishanu Chakraborty, Shreya More, Isha Fuletra and Anushka Sharma – as well as Puneet Kaur and Lenny Tin.
This article was originally published on IDinsight.
The second article in this series can be found here.
