Best practices for conducting phone surveys

COVID-19 has accelerated the trend towards a greater number of phone surveys for data collection activities in the development sector. There are many benefits to phone surveys outside of just minimizing health and safety risks, including reduced cost and greater scalability. However, a core challenge for phone surveys is capturing high-quality, rich data while also seeking to uphold the dignity and trust of respondents. 

A core best practice to phone survey techniques is building a strong rapport with the respondent, and using a respondent’s name is very helpful in this trust-building process. So when we were faced with a situation recently where we did not have access to the names of the people we were calling, we were rather concerned that this may negatively affect our ability to collect high-quality data and maintain a positive relationship with respondents. 

However, we found that a lack of respondent names need not be an inhibiting factor for conducting detailed, personal, phone surveys. There are of course many survey techniques that do not require respondent names. For example, random digit dialing and many large-scale survey techniques regularly do not have respondent details including names. For this survey we were conducting a long-form survey that asked personal details on energy use and household characteristics.

Background to our survey

Last year, we completed over 1,700 phone-surveys with households and businesses in Bihar and Uttar Pradesh, India. The respondents all received distributed renewable energy (DRE) from a variety of energy service companies (ESCOs) and we wanted to learn more about their energy usage, energy preferences and household decision-making. The data from the survey was used to build a baseline for a long-term monitoring system. 

In order to complete the survey, the ESCOs shared with us a list of their customers with their customer type and phone number, but not their names. For very justifiable reasons, these private companies wanted to ensure the confidentiality of their customers. So we built a surveying approach that aimed to build rapport with customers in spite of not having names. 

We had two hypotheses going into the process:

1. Identification of the correct respondents would be more difficult; and
2. Survey refusal rates would increase due to lack of trust and rapport between enumerators and respondents.

To test these hypotheses and develop best practices for future surveys, we collected data from both respondents and enumerators.¹

Hypothesis 1: Identification of respondents would be more difficult

A challenge of surveying without names of respondents is confirming we are talking to the correct person. It was possible that the phone numbers we received from ESCOs either changed, or the person who answered the phone was not the intended respondent. 

To attempt to overcome this challenge, we developed a four-stage triangulation process that we went through in each survey.  

We observed the following: 

• About 11 percent of respondents did not match with the Electricity Company (ESCO) we had in our database. Upon deeper investigation which included speaking with enumerators, reviewing audio and calling back certain respondents, we found that many of these respondents simply did not know their ESCO’s names but knew they had a ‘solar connection’. This led us to ask respondents who did not know their ESCO whether they had a ‘solar connection’ which was ‘non-government’ provided.   
• About 98 percent of the respondents reported residing in the same state as reflected in our database.
• We compared responses of these respondents with those who easily identified their ESCO’s name on basic questions such as electricity bill amounts and year of connection. For example, the median bill amounts varied by ~100 INR only. 
• Nearly 20% of respondents who were unable to identify their ESCOs also hesitated in sharing their names. On the flip side, only two percent of respondents who were able to identify their ESCO hesitated to share their names.

Learnings and Tips

• Leverage all the data available to you to triangulate and attempt to confirm that you are speaking with the right respondent. 
• Consider non-direct methods for identifying respondent characteristics. These methods may not result in certainty that you are speaking with the right respondent but will give you greater confidence. 
• We observed a high-level of correlation between refusal to share names and inability to name their energy company. This may suggest that when we contact the incorrect respondent, they will be more hesitant to share their name or other personal information.   

Hypothesis 2: Survey refusal rates would increase due to lack of trust and rapport between enumerators and respondents

Another potential issue of surveying without names is gaining trust with the respondents. Respondents may feel awkward that someone is calling them to request personal information and does not know the person’s name they wish to speak with. 

To our surprise, this was less of an issue than expected! Over 95% of respondents willingly shared their names at the start of our calls after we received consent. 

We did note that of the 45 respondents who hesitated in providing their names, 12 (or 26 percent) refused to continue the survey after a few modules or cut the call without providing any reason. On the other hand, of the 1058 respondents who easily shared their names, 154 (or 14.5 percent) refused to continue the survey after a few modules. 

Learnings and Tips 

• Including clear, concise but complete consent language at the start of the call is critical. We also recommend investing time to ensure that translation of the consent language is high-quality. 
• Provide a clear explanation for why we lack names and why we are asking. 
• Get support from trusted partners. We noted that responses are more likely if a trusted authority can let respondents know they may be contacted and why the call is valuable. Additionally, it can be helpful if the field team of the organization is aware that a survey is being conducted, and can support by responding to customer queries.²
• Confirm at the end of the call that personally identifiable information (including names) will not be shared with anyone externally, and also provide an opportunity for either direct or indirect feedback on the survey. 

Our experience provided a new lens on the need of PII for conducting detailed phone surveys. Overall, there are certain challenges while surveying respondents without their names. These include risk of incorrectly identifying the intended respondent, difficulty in establishing trust with respondents and respondents refusing to complete the survey. However, these challenges may be mitigated to a large extent by using other information available, comparing responses of known and unknown respondents, coordinating with field functionaries to inform respondents about a survey exercise and using a strong consent script.

—

Footnotes:

1. It is important to note that we are unable to make any causal statements on how our processes may have led to certain observed outcomes. We do not have a counterfactual situation, so our learnings are descriptive and qualitative in nature.
2. However, the partner organisation’s field team should not have access to the questionnaire, which they might use to bias the respondents’ answers. In our situation, they did not have a sense of what topics we planned to cover in this survey.

This article was originally published on IDinsight.