{"id":1374,"date":"2019-07-19T11:30:14","date_gmt":"2019-07-19T11:30:14","guid":{"rendered":"https:\/\/idronline.org\/2020\/12\/23\/single-studies-cannot-inform-policy-making\/"},"modified":"2025-11-10T06:59:16","modified_gmt":"2025-11-10T01:29:16","slug":"single-studies-cannot-inform-policy-making","status":"publish","type":"post","link":"https:\/\/idronline.org\/kn-in\/single-studies-cannot-inform-policy-making\/","title":{"rendered":"Single studies cannot inform policy-making"},"content":{"rendered":"

For a development evaluator, the holy grail is to have evidence from one’s study be taken up and used in policy or programming decisions that improve people’s lives. It’s not easy. Decisions are based on many factors. The availability of evidence is just one of them. And of course, even when evidence is taken up, it does not mean that it will lead to the right decision.<\/p>

A dramatic example of the complexity of take up is the political and judicial controversy that has swirled in the Philippines recently. The events could lead to a prominent researcher and senior officials (including a former health minister) being prosecuted precisely for acting on evidence of impact. The problem is that they are alleged to have done so a bit prematurely.<\/p>

Dengue is a mosquito-borne disease that threatens the health and sometimes life of millions in many countries, including here in India. In 2014, an article<\/a> published in The Lancet reported the efficacy of a new vaccine, Dengvaxia, in dramatically lowering the incidence of the disease. The drug was approved in 19 countries<\/a>, including the Philippines. In late 2016, the Philippines procured millions of doses and, over the span of 18-24 months, proceeded to inoculate some 800,000 school children.<\/p>

\"What<\/a><\/div>

\n

\"A

Picture courtesy: Wikimedia Commons<\/p><\/div>\n<\/p>

Related article: Do evaluations actually serve a useful purpose?<\/strong><\/a><\/p>

Arguably, the original study ticked many of the boxes for success. Its findings were taken up and turned into policy that could improve lives. An added bonus was that the lead author of the article, which had passed the most rigorous of peer reviews, was from a low- and middle-income country, the Philippines.<\/p>

But life is seldom so simple. Recent articles in Science<\/a> and Scientific American<\/a> document why this study has stirred up such controversy in the country. There are now criminal charges being considered against the researcher and health officials, who sanctioned the programme. While the clinical trials were apparently well-conducted, there is doubt about how policies were drawn from them. Apparently, the vaccine, while very effective as a prophylactic for those who have already been exposed to dengue, may actually be dangerous for those who have never had it. Subsequent studies<\/a> may be used by prosecutors to imply that policy influence can even be deadly.<\/p>

It would be a shame if this example disincentivises decision makers from using evidence. Especially these days, when the standards for evidence seem under attack, rigorous research cannot be neglected. It wasn’t long ago that another vaccine study<\/a>, now discredited, suggested a link between the measles, mumps, and rubella vaccine and autism. Although the study<\/a> has been debunked, it continues to fuel vaccine hesitancy across Europe. This is why it’s so important to assess when and how evidence from rigorous studies should be taken up. At 3ie, our team has been using such experiences to draw lessons.<\/p>

When there is not enough evidence to synthesise, it is important to generate more evidence.<\/p><\/blockquote>

One lesson is the danger of relying on single studies to inform policy. Multiple contextual factors can influence effectiveness. To the extent possible, synthesise evidence through rigorous theory-based systematic reviews that include meta-analysis. 3ie is not only supporting this work but it is also facilitating access to evidence through our systematic review repository<\/a>.<\/p>

\"donate<\/a><\/div>

When there is not enough evidence to synthesise, it is important to generate more evidence. For example, 3ie has an evidence <\/a>programme<\/a> that is supporting studies testing innovative approaches in engaging communities for improving low and stagnating immunisation rates across different contexts. Study findings are showing that in low- and middle-income countries, there are important practical barriers to immunisation. Lack of information on when and where to get children immunised, inadequate information on immunisation, travel costs and the long waits at health centres, and a dearth of health staff and services, are big challenges.<\/p>

A final lesson is the importance of promoting research transparency and replications. The public inquiry into the Dengvaxia episode was prompted by a healthy scholarly debate which included policymakers. A critical aspect is the replication of scientific work. The ‘crisis’ caused by the inability to replicate some highly influential studies (including in the social sciences) is beginning to change how research is done<\/a>. 3ie has been supporting this work through its replication <\/a>programme <\/a>and by our strong commitment to improving transparency.<\/p>

Hopefully lessons like these on the use of rigorous evidence will be helpful, especially in a world when facts are so easily distorted.<\/p>

With inputs from Radhika Menon. This article was originally published<\/a> on 3ie.<\/em><\/p>\n

\"donate<\/a><\/div>","protected":false},"excerpt":{"rendered":"

For a development evaluator, the holy grail is to have evidence from one’s study be taken up and used in policy or programming decisions that improve people’s lives. It’s not…<\/p>\n","protected":false},"author":8,"featured_media":1375,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[9,582,15],"tags":[3691],"series":[],"meta-filter":[639],"schema-filter":[743],"no-display":[],"class_list":["post-1374","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-health","category-instagram","category-monitoring-evaluation","tag-programme-design","contributor-emmanuel-jimenez","states-india","meta-filter-curated-article","schema-filter-article"],"yoast_head":"\nSingle studies cannot inform policy-making - India Development Review<\/title>\n<meta name=\"description\" content=\"While using evidence is important to form policies and programmes, it may not always lead to the right decision, particularly when there is not enough material.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/\" \/>\n<meta property=\"og:locale\" content=\"kn_IN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Single studies cannot inform policy-making\" \/>\n<meta property=\"og:description\" content=\"While using evidence is important to form policies and programmes, it may not always lead to the right decision, particularly when there is not enough material.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/\" \/>\n<meta property=\"og:site_name\" content=\"India Development Review\" \/>\n<meta property=\"article:published_time\" content=\"2019-07-19T11:30:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-10T01:29:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/idronline.org\/wp-content\/uploads\/2020\/12\/A-child-being-given-polio-vaccine_wikimedia-commons.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"681\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Devanshi Vaid\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Single studies cannot inform policy-making - India Development Review","description":"While using evidence is important to form policies and programmes, it may not always lead to the right decision, particularly when there is not enough material.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/","og_locale":"kn_IN","og_type":"article","og_title":"Single studies cannot inform policy-making","og_description":"While using evidence is important to form policies and programmes, it may not always lead to the right decision, particularly when there is not enough material.","og_url":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/","og_site_name":"India Development Review","article_published_time":"2019-07-19T11:30:14+00:00","article_modified_time":"2025-11-10T01:29:16+00:00","og_image":[{"width":1024,"height":681,"url":"https:\/\/idronline.org\/wp-content\/uploads\/2020\/12\/A-child-being-given-polio-vaccine_wikimedia-commons.jpg","type":"image\/jpeg"}],"author":"Devanshi Vaid","twitter_card":"summary_large_image","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/#article","isPartOf":{"@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/"},"author":{"name":"Devanshi Vaid","@id":"https:\/\/idronline.org\/#\/schema\/person\/5440619b9bec1184c5774c076c0ca71c"},"headline":"Single studies cannot inform policy-making","datePublished":"2019-07-19T11:30:14+00:00","dateModified":"2025-11-10T01:29:16+00:00","mainEntityOfPage":{"@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/"},"wordCount":790,"commentCount":0,"publisher":{"@id":"https:\/\/idronline.org\/#organization"},"image":{"@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/#primaryimage"},"thumbnailUrl":"https:\/\/idronline.org\/wp-content\/uploads\/2020\/12\/A-child-being-given-polio-vaccine_wikimedia-commons.jpg","keywords":["Programme design"],"articleSection":["Health & Well-being","Instagram","Monitoring & Evaluation"],"inLanguage":"kn-IN","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/","url":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/","name":"Single studies cannot inform policy-making - India Development Review","isPartOf":{"@id":"https:\/\/idronline.org\/#website"},"primaryImageOfPage":{"@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/#primaryimage"},"image":{"@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/#primaryimage"},"thumbnailUrl":"https:\/\/idronline.org\/wp-content\/uploads\/2020\/12\/A-child-being-given-polio-vaccine_wikimedia-commons.jpg","datePublished":"2019-07-19T11:30:14+00:00","dateModified":"2025-11-10T01:29:16+00:00","description":"While using evidence is important to form policies and programmes, it may not always lead to the right decision, particularly when there is not enough material.","inLanguage":"kn-IN","potentialAction":[{"@type":"ReadAction","target":["https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/"]}]},{"@type":"ImageObject","inLanguage":"kn-IN","@id":"https:\/\/idronline.org\/single-studies-cannot-inform-policy-making\/#primaryimage","url":"https:\/\/idronline.org\/wp-content\/uploads\/2020\/12\/A-child-being-given-polio-vaccine_wikimedia-commons.jpg","contentUrl":"https:\/\/idronline.org\/wp-content\/uploads\/2020\/12\/A-child-being-given-polio-vaccine_wikimedia-commons.jpg","width":1024,"height":681,"caption":"Picture courtesy: Wikimedia Commons"},{"@type":"WebSite","@id":"https:\/\/idronline.org\/#website","url":"https:\/\/idronline.org\/","name":"India Development Review","description":"India's first and largest online journal for leaders in the development community","publisher":{"@id":"https:\/\/idronline.org\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/idronline.org\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"kn-IN"},{"@type":"Organization","@id":"https:\/\/idronline.org\/#organization","name":"India Development Review","url":"https:\/\/idronline.org\/","logo":{"@type":"ImageObject","inLanguage":"kn-IN","@id":"https:\/\/idronline.org\/#\/schema\/logo\/image\/","url":"https:\/\/idronline.org\/wp-content\/uploads\/2021\/01\/idr-logo-without-tagline-sm.webp","contentUrl":"https:\/\/idronline.org\/wp-content\/uploads\/2021\/01\/idr-logo-without-tagline-sm.webp","width":1,"height":1,"caption":"India Development Review"},"image":{"@id":"https:\/\/idronline.org\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/idronline.org\/#\/schema\/person\/5440619b9bec1184c5774c076c0ca71c","name":"Devanshi Vaid","image":{"@type":"ImageObject","inLanguage":"kn-IN","@id":"https:\/\/secure.gravatar.com\/avatar\/2ff7c808d3539c16c67949df0296e034b2baeb599fa173f683398d23fe4e3690?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/2ff7c808d3539c16c67949df0296e034b2baeb599fa173f683398d23fe4e3690?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2ff7c808d3539c16c67949df0296e034b2baeb599fa173f683398d23fe4e3690?s=96&d=mm&r=g","caption":"Devanshi Vaid"},"url":"https:\/\/idronline.org\/kn-in\/author\/dvaid\/"}]}},"_links":{"self":[{"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/posts\/1374","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/comments?post=1374"}],"version-history":[{"count":4,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/posts\/1374\/revisions"}],"predecessor-version":[{"id":11760,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/posts\/1374\/revisions\/11760"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/media\/1375"}],"wp:attachment":[{"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/media?parent=1374"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/categories?post=1374"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/tags?post=1374"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/series?post=1374"},{"taxonomy":"meta-filter","embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/meta-filter?post=1374"},{"taxonomy":"schema-filter","embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/schema-filter?post=1374"},{"taxonomy":"no-display","embeddable":true,"href":"https:\/\/idronline.org\/kn-in\/wp-json\/wp\/v2\/no-display?post=1374"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}