Six myths about remote data collection I India Development Review
September 30, 2020

Six myths about remote data collection

A cost-effective approach that is more than just a substitute for in-person surveys.

8 min read

The COVID-19 crisis has affected nonprofit fieldwork, in some cases limiting it considerably. Because of this, many teams may have had to reconsider how to proceed with collecting data. In this piece, we take a close look at remote data collection, a relatively new approach that employs technology as a substitute to on-site or in-field surveyors.

At this time, transitioning to remote data collection may seem like an appealing choice for nonprofits. But before going down this path, it’s important to consider a few things. We underscore that this should only be done after considering the suitability of this option from a contextual and ethical perspective. First, remote data collection inherently relies on connecting with people through some form of technology, thereby excluding the most marginalised, as they may have limited infrastructural connectivity. Second, collecting data remotely is not a good idea if it puts anyone at greater risk of infection (for example, through shared phones), or if it increases the likelihood of participants facing domestic or other violence. Lastly, at this time of crisis, remote data collection shouldn’t be considered if it takes up a lot of time that would otherwise be spent on tasks critical to health and livelihoods, such as farming or travelling to seek care.

Related article: How to get the most out of data

When collecting remote data does not raise any of these flags, this relatively new approach presents some exciting possibilities. Below, we’ve compiled a list of common myths about remote data collection that we bust, while mentioning important caveats.

Myth 1: Digital data collection is remote data collection

Digital data collection and remote data collection often get mistakenly interchanged but are distinct concepts. The former doesn’t have to be done remotely. The latter does, and often involves some digitisation of the process.

Digital data collection is simply any data collection that has moved away from traditional paper-based formats. It is also commonly known as CAPI or Computer Assisted Personal Interviewing and relies on the relative efficiency of electronic surveys and built-in algorithms for faster and more accurate data collection and data entry. Digital data collection thus enables surveyors to enhance their data collection efforts using preloaded electronic surveys on phones or tablets. These surveys can be done either in-person or remotely.

Remote data collection, on the other hand, uses a wide array of technologies to gather data when it’s not possible to do so in-person.

The transition to digital data collection in the global development community over the past decade or so has mitigated some of the limitations of paper-based surveys such as human errors with data entry, navigating survey-skip patterns in complex questionnaires, and monitoring surveyor performance from afar.

Remote data collection, on the other hand, uses a wide array of technologies to gather data when it’s not possible to do so in-person (more on the technologies in Myth 6). These cater to different needs, depending on the nature of data being collected, the organisation’s capacity, and the available telecom infrastructure.

Myth 2: All data gathering planned pre-COVID-19, must now happen remotely

Remote data collection cannot simply replace all the in-person data collection planned prior to COVID-19. During a pandemic such as the one we are experiencing now, we need to consider the need and suitability of collecting data, especially from people who may be severely affected by the socio-economic effects of the crisis.

There are a set of fundamental ethical and technical questions to answer before proceeding with data collection. Of these, the most important is the question of whether collecting this data will improve respondents’ lives in the next few months during the COVID-19 crisis. If not, it can be put on hold until it is feasible to move ahead safely and effectively.

Man on phone taking notes_Rawpixel

Remote data collection inherently relies on connecting with people through some form of technology, thereby excluding the most marginalised, as they may have limited infrastructural connectivity. | Picture courtesy: Rawpixel

When it comes to practical considerations for gathering data remotely, here are some questions to ask yourself before you begin:

  • Do my target respondents have reliable mobile network coverage?
  • Are they likely to be comfortable having long conversations over the phone? If not, will I really be able to ask what I need to in a short questionnaire that can be administered remotely, amidst a pandemic?
  • What type of data am I collecting, and will I be able to trust its accuracy and reliability when collected remotely? To answer this question, consider whether the data collectors already enjoy a good rapport with the respondents, and if not, whether they will be able to build it over the phone. In addition, consider the best practices and emerging evidence for your specific domain of work. For example, in the case of researching sensitive topics such as violence against women, there have been some nuanced findings on whether gathering data remotely is in fact harmful or helpful.

Related article: How to use data to improve decision-making

Myth 3: Only consider collecting data remotely when doing so in-person is not possible

In certain scenarios, gathering data remotely can be more effective than face-to-face interviews:

1. When your respondents are difficult to reach in-person, especially during the day

In settings with minimal connectivity issues, phone surveys save time and money on travel. For example, a Bangalore-based social enterprise found that its survey team spent hours in traffic to reach respondents who worked across the city and could only talk to them on a tea break. To reduce the logistical costs and mitigate interviewer fatigue, the research teams experimented with phone surveys and found them to be a more suitable option for data collection.

2. When collecting routine monitoring data related to programme progress, reach, or satisfaction

With remote surveys, you can collect data frequently without additional travel expenses. Apart from surveys, checklists and attendance registers are examples of other tools that lend themselves to remote methods of data collection.

3. When your programme itself is remote in nature

If your programme is delivered remotely (for example, you run a helpline or offer training and educational content online), collecting data remotely might be preferable to doing so in-person. First, there is a good chance that your respondents are comfortable with the technology that you use to deliver the programme, and so it can also be used for data collection with relative ease. Second, it’s possible that respondents are in different locations, making a remote set up more economical.

Myth 4: Only quantitative data is conducive to remote data collection methods

People generally view qualitative data collection as a more involved and lengthier process than quantitative data collection—one that relies on trained interviewers with unique skills. But there are ways to collect qualitative data remotely too.

Two key considerations to bear in mind (more caveats should be reviewed here):

1. Choice of qualitative method

Generally, one-on-one interviews are both logistically and technically more conducive to collecting data remotely, compared to focus group discussions (FGDs). You can conduct interviews over the phone as long as both parties have connectivity. FGDs in contrast, ideally require video and phone capability among all participants. Plus, it’s harder to ensure a quiet and comfortable environment across multiple participants, than it is for one. On the technical front, conducting an FGD remotely makes the role of the moderator more challenging, given the absence of non-verbal cues during a video call.

2. Target group for data collection

In remote settings, phone interviews will likely yield richer data if respondents are already familiar with the interviewers. Additionally, some target groups may be more comfortable with and used to being asked their opinions. Interviews with government officials, nonprofit staff, or community leaders, for instance, are likely to be conducive to qualitative remote data collection.

Myth 5: Collecting data remotely is expensive

The technology and method used for this process will determine the final cost. Plus, regardless of technology choices, you are likely to save costs because you won’t need the field machinery and logistics for in-person interviews, especially in the case of hard-to-reach areas.

For larger and more complex programme evaluations, a popular software many use is CATI or Computer Assisted Telephonic Interviews.

Organisations can choose from a wide range of technology aids across different price points to support their data collection. For larger and more complex programme evaluations, a popular software many use is CATI or Computer Assisted Telephonic Interviews. CATI offers the benefits of digital data collection but adapted for use over telephonic interviews. It requires a well-trained cadre of enumerators collecting data digitally, on phones or other hand-held devices, such as tablets. The main costs include the CATI software subscription costs, data collection devices, and human resources required for training, monitoring, and collecting the data. Depending on budgets, partnering with firms experienced with remote data collection, such as IDinsight or Outline India, is a good option to manage the entire process.

Less expensive alternatives involve using open-source CATI software or web-based survey tools and trained phone surveyors. Organisations can invest in training their own staff using resources available online from experienced practitioners of remote data collection (listed at the end of this article) and through training webinars offered by vendors such as Survey CTO.

If you can’t carry out in-house training, you can use third-party recruiters, or call centres. You can also hire trained surveyors from remote work platforms such as Fieldscope, a social enterprise founded by one of the authors of this article, or freelance platforms such as Upwork. Regardless of who makes up the data collection team, consider compensating them for air-time and other logistical expenses if they’re using their own devices.

Related article: Using data to improve programme efficiency

Myth 6: Phone calls are the only channel to collect data remotely

In the absence of face-to-face interviewers, video calls or phone calls are often considered the next best substitute for quality data collection. The choice of medium however ultimately depends on your programme and the nature of data being collected. You can choose from a number of channels, based on how appropriate they may be for the nature of your questions. This includes phone surveys, SMS, Interactive Voice Response (IVR), and online surveys.

SMS and IVR may be suitable for simple and short questions, on topics that aren’t sensitive. These are relatively new methods to measure socio-economic indicators, especially among the poor, and evidence on how accurate these techniques are is nascent. For example, a study carried out by Mark Schreiner compared IVR and phone-data collection methods to in-person interviews (considered the benchmark for measuring poverty rates using the Progress out of Poverty Index). Now known as the Poverty Probability Index, this survey is used by hundreds of organisations in about 60 countries. In this study, Schreiner found that poverty estimates varied across the different data collection methods for the same population; flagging the potential for variability in results when using different data collection channels.

Apart from this, in settings where respondents have internet connectivity, online or web-based surveys have long been used as a fast and cost-effective method for remote data collection. Popular options include Survey Monkey, Google Forms, and Typeform. They are suitable for quick assessments, for understanding community needs or collecting feedback on the performance of a product or service.

 The ‘cheat sheet’ below from 60 Decibel’s Remote-Survey Toolkit is a good starting point to compare options.

remote data collection cheat sheet_Remote Survey Toolkit by 60 Decibels

Source: Remote Survey Toolkit by 60 Decibels

Remote data collection and its supporting ecosystem have evolved to offer a viable solution for data that must be collected and can be collected ethically in lieu of face-to-face methods. Organisations should carefully consider the methodology and tools based on the nature of their programmes and the purpose of data collection. Finally, teams must share their experiences with such techniques and learn from one another, so that we can strengthen our collective ability to adapt to the changing needs of our sector.

Know more

Do more

  • Share best practices and lessons learned of your remote data collection experiences through blogs, videos, panel discussions, and other channels to strengthen the sector’s knowledge of this emerging approach.
  • Advocate sectoral and organisational leadership for upskilling and investing in talent to adapt to new technologies and methods for the post-COVID-19 era.
  • Attend relevant online training or webinars and complete courses such as Acumen’s Lean Data, online training offered through Survey CTO on CAPI, among others.
We want IDR to be as much yours as it is ours. Tell us what you want to read.
ABOUT THE AUTHORS
priya kekre_profile
Priya Kekre

Priya Kekre is a global health researcher and social entrepreneur. She has extensive experience in programme design and measurement specifically with community health issues in low-resource settings. She’s the co-founder of Fieldscope, an online platform that catalyzes on-demand skilled collaboration for the social sector. Fieldscope’s diverse user base includes development practitioners and researchers from Columbia University, the World Bank, Oxford Policy Management, IDinsight, One Acre Fund, and Haqdarshak, among others. Priya has a master’s degree in Global Public Health from Emory University, USA.

Devyani Srinivasan profile
Devyani Srinivasan

Devyani Srinivasan has been an independent research and M&E consultant for more than a decade. As an M&E consultant, she focuses on: 1) collaborating with her clients to formulate frameworks that include logic models/theories of change and indicators, 2) conducting evaluations, and 3) providing training on M&E. Her clients include Nilekani Philanthropies, Oxfam in India, and IFAD in Afghanistan and Kenya. She possesses a bachelor’s degree from Wesleyan University in the U.S and an MSc in Planning from the University of Toronto in Canada.

COMMENTS
READ NEXT