The COVID-19 pandemic forced our team at SNEHA to transition to a remote system of data collection for research and evaluation. Traditionally, face-to- face interviewing (where there is a greater connect between the surveyor and the respondent) is believed to deliver better results. However, due to the safety concerns posed by the current crisis, we needed to embrace phone surveys as a norm for collecting data.
As such, we have conducted more than eight phone surveys over the last 10 months, both to implement our evaluation framework and to enhance our understanding of community responses to COVID-19. After having surveyed more than 3,500 people, here’s what we have learned from the process.
1. Build rapport
Transparent communication and building a trust-based relationship with the respondent is crucial in primary data collection. On the field, surveyors can establish their identity with an identity card or letter on organisational letterheads. This becomes difficult during phone interviews; making it is essential to engage respondents meaningfully in the initial minutes of the conversation. Explaining how the interviewer obtained the respondent’s number and the purpose of the survey is important, as is building a rapport and sustaining their interest in the conversation.
2. Take informed consent
One of the first things we learn about data collection is the importance of obtaining informed consent from respondents. To do this, interviewers need to ensure that respondents fully understand the objective of the data collection process and what is expected of them. A few key points to include in communicating this are:
- The expected duration of the interview (asking for their time upfront helps avoid situations where the respondent disconnects mid-interview)
- The purpose of the interview and the type of questions that will be asked
- In the case of adolescents, parental buy-in
3. Create a flexible call-back protocol
When planning how often we would reach out to respondents, our team decided on a protocol that included three phone calls to a participant a day, preferably with an interval of two to three hours, over a period of three days. Despite this, we needed to make changes as per the convenience of the respondents and their context. For example, in one of our intervention areas, we received a low response for calls made to women in the morning because in that area water was supplied to people’s homes in the mid-morning hours. In another area, we found that most women rested after their morning chores and weren’t inclined to give interviews in the afternoon. Taking these realities and local context into account, allowed us to tweak our systems in accordance with them.
4. Keep the interview short
An ideal phone interview, as emphasised by researchers, should not exceed twenty to thirty minutes. This timeframe becomes all the more essential during a pandemic where respondents are likely to be juggling many responsibilities at once. To stick to this window, it is important to look at which questions in the survey take the most time to answer. For example, in our experience, we found that asking too many socio-economic and demographic questions (such as listing the household assets to calculate wealth index) increased the time taken to complete the interview. Being cognisant of this allowed us to weed out questions unrelated to the research or evaluation objectives.
5. Ensure data quality
To ensure the quality and accuracy of the data collected over the phone, organisations should think of factors such as accuracy of person being reached and whether they have the privacy needed to answer honestly. In our experience, to be certain that the target respondent was being reached, we added a few probing questions at the beginning of the survey and gave interviewers a list of supplementary information for added verification. This included details such as specific home/street address or in the case of pregnant women, their gestational history, which only they would know.
Surveyors made sure to ask respondents whether they were comfortable to continue talking.
With experience, our surveyors became skilled at identifying non-verbal cues from respondents (voice modulation, identifying surrounding sounds or echoes if the phone has been kept on speaker mode). In addition, surveyors made sure to ask respondents whether they were comfortable to continue talking, and if not, they rescheduled the interview—a practice which helped us maintain both the quality of the data and response rates. Lastly, in addition to instituting software checks to prevent data entry errors, we created drop down menus for responses in the mobile application. This helped eliminate errors and save time.
6. Build surveyor capacity and keep an open line of communication
Despite years of experience, our surveyors were initially apprehensive about phone surveys. In addition to worrying about whether they would be able to meet their targets, they were concerned about gauging a respondent’s comprehension during the interview, and about asking the more sensitive questions around family members.
We took interviewers through possible scenarios they might encounter.
Given this, our team had interviewers conduct a phone survey with the SNEHA staff before they began speaking to community members. In addition, we reminded them that response rates would be lower for phone surveys because of which the practice of setting daily targets was removed. Lastly, we took interviewers through possible scenarios they might encounter—for example, if a respondent sounded depressed or if there was a need expressed for ration or medical services, appropriate referral linkages were explained to them.
Based on our experience during the pandemic, we recommend remote data collection for surveys with stakeholders such as volunteers, health systems staff, and community leaders. This is especially true given the fact that remote work allowed us to conduct frequent online debriefing team meetings where any challenges were instantly addressed and co-learning was made possible.
We know that phone surveys can never fully substitute face-to-face interviews, especially when anthropometric and other measurements need to be captured. However, there is place for remote data collection, and as such, we as a sector need to build our capacity to make the most of it.
The views expressed here are based on inputs received through a qualitative exploration (three in-depth interviews and two focus-group discussions) with the monitoring and evaluation team at SNEHA, including a Monitering, Evaluation, Accountability, and Learning (MEAL) specialist, two supervisors and eleven surveyors.
—
Know more
- To better understand how SNEHA conducts community surveys, check out their resources here.
- To learn more about how to hire and train and phone survey team, identify the best length for a phone survey, or seek consent during qualitative and quantitative interviews, check out these resources by IDinsight.
