Course 5: Global Health Estimates

Introduction

Despite rapid advances in technology and medicine in today’s globalizing world, many low-and middle-income countries continue to lack the necessary tools and resources required for measuring the vital health statistics of their citizens. Due to the ineffective and underfunded nature of civil registration systems in these areas, political leaders and public health official are forced to seek global health estimates to compensate for non-existent or unreliable data.(2) These estimates are typically derived from statistical models that utilize mathematical and scientific predictions to impute values where data is missing or unavailable. (1) This type of estimation work is typically conducted, funded, and published either by agencies under the direction of the United Nations, such as the World Health Organization, or by private academic institutions. Incongruous estimations and competing statistics published by the dichotomous groups have sparked debate between experts over not only who is more adequately suited to generate global health estimates, but also the accuracy of the estimates.(3) Accuracy is critically important to public health planning and prevention since the estimates are used to determine the allocation of public resources, which has a strong influence over health policy. Thus, the United Nations and academic institutions alike are placing tremendous emphasis on improving the accuracy of global health estimates while also seeking ways to improve data collection in resource-poor countries. (4) 

Benefits of Global Health Estimates

Despite the controversy, health estimates continue to play an important role in shaping global health initiatives and prevention strategies. In the absence of actual data, estimates provide a framework to guide decision making and policy planning that would be simply non-existent without such estimates.(5) Ideally, there eventually should be a decreasing reliance on global health estimates as low-and middle-income countries strengthen their capacity to collect, archive, and access vital health statistics.(6) In the meantime, estimates provide at least partial compensation for the lack of data in resource-poor countries. In fact, using a specific set of numbers to define the prevalence of a health problem or disease, whether derived from an estimate or based on an actual measurement, helps to quantify the magnitude of an issue. Thus, estimates play a crucial role in drawing attention to global health challenges that may have otherwise gone unacknowledged or underestimated.(7) Additionally, once a health estimate has been published, either by the UN or by an academic institution, the publication is likely to incite public support for the issue and inspire further research on the subject.

The potential merits of global health estimates extend beyond the realm of public health and into the global economics arena. Global health estimates serve as an important economic benchmark that help to guide policymakers and health officials towards more fiscally-sound spending decisions.(8) Health care spending currently represents over 10% of the world’s economic output, and as it continues to assume a growing role in the global economy, it is increasingly important that the UN and other governing bodies allocate health resources judiciously.(9) Estimates of the prevalence, cost, and impact of a disease equip policymakers with the information necessary to analyze and select the most appropriate planning and prevention strategies, and ultimately these decisions have the potential to save lives and money.(10)  Radboud Duintjer Tebbens, an advocate of the Global Polio Eradication Initiative (GPEI), explains: “Polo eradication is a good deal, from both a humanitarian and an economic perspective. The GPEI prevents devastating paralysis and death in children and allows developing countries and the world to realize meaningful financial benefits.”(11) Eradicating the transmission of wild polioviruses by 2015 could result in a global net savings of 40 to 50 billion dollars if the GPEI’s estimates are correct.(12)

Challenges of Global Health Estimates

Despite these benefits, it is vitally important to recognize that the value global health estimates is contingent upon the accuracy of the estimates.  Herein lies the most rudimentary challenge associated with global health estimates: an estimate is rarely considered as accurate and reliable as actual data, assuming that the data itself is correct and has been appropriately processed.(13) With the right tools, technology and information, some estimates generate figures that are quite dependable. However, many estimates prove to be highly unreliable as researchers continue to use increasingly complex statistical models in an attempt to compensate for missing data. Eventually, the data used to develop these estimates becomes completely over-worked, producing measurements that scarcely resemble the original input.(14) Greater degrees of statistical and methodological complexity reflect increasing levels of uncertainty and decreasing probabilities of accuracy. “The complexity of such models often masks a stark reality: most people are born and die uncounted, the reasons for their deaths unknown.”(15) Despite this reality, estimates generated using overly intricate models are often reported as actual facts rather than estimates: “Literature is relatively scare on the extent and quality of primary sources of health data on a global scale, even though there are many reports that present aggregated global data on various health issues, sometimes giving the impression that those estimates carry a high degree of certainty.”(16) Mistaking estimates as more reliable than they actually are poses a dangerous threat to the efficacy of public health planning and policy, making the inaccuracies of global health estimates a considerable challenge.

The inappropriate use of statistical modeling is just one of many factors contributing to the erratic nature of global health estimates. An additional problem hindering estimation accuracy is the lack of standards and regulations dictating how specific health statistics should be measured, stored, and shared.(17) Furthermore, there are no uniform standards or training procedures in place to ensure that technology designed to collect vital health statistics are being utilized properly. This lack of standards often allows for biased data collection, especially in developing countries. For example, only a small minority of the population in low-income countries is likely to use modern medical services. Thus, vital registration systems in these areas are more likely to record the health information of a country’s wealthiest and best educated citizens rather than a more representative sample of the entire population. Additionally, health care facilities in resource-poor countries are typically reserved for the most severely ill patents and so these people are far more likely to be formally accounted for than other citizens.(18) The development and enactment of international regulations to guide data collection is critically important for obtaining high quality information, yet there is debate over who should set such regulations. Many experts believe that the World Health Organization or the United Nations is best equipped to set universal standards.(19)   However, others groups worry that the United Nation’s involvement in collecting and generating global health estimates would compromise it’s ability to set unbiased standards: “It is difficult to see how the UN can be the trusted developer of norms and standards and a respected neutral broker when it is also trying to compete in the health measurement arena.”(20) Determining who is best suited for overseeing the creation of universal standards and how those standards should be implemented is a particularly dicey challenge, one that is further complicated by the fact that not all countries possess the same resources and technology for collecting vital health information making it especially difficult to regulate uniformly.

The controversy over implementing norms and regulations represents a small piece of a larger debate over who should be conducting health research and generating global health estimates. The two major players in health estimation are currently the UN and academic institutions. Proponents of UN-generated estimates argue that academic institutions produce biased figures because they rely largely on previously existing research and studies, using mixed methods of statistical analysis to generate estimates. This method is considered problematic; if the information from the previously published source is an estimate, then new estimates are being generated from older estimates, rather than from actual data. Additionally, since academics institutions are independent from governing bodies, they have no obligation to consult the countries whose statistics are being reported.(21) These issues undoubtedly have a negative impact on the overall quality of global health estimates produced by academic institutions. Similarly, the UN faces significant criticism over its production of global health estimates, though for different reasons. Global health estimates generated by the United Nations or the World Health Organization are sometimes inappropriately influenced by politics. The UN has been accused of knowingly publishing inaccurate figures in order to avoid confrontation with other government authorities.(22) Chris Murphy, the president of the Institute of Health Metrics and Evaluation explains: “Specific countries will be unhappy with figures that the WHO or another agency are suggesting and put marked pressure to have them changed or to modify them. This happens all the time.”(23)

The issues facing both academic institutions and the United Nations have the potential to seriously compromise the validity of the global health estimates produced by each group. Rather than working together to minimize their imperfections and maximize their strengths, the UN and academic institutions conducting estimation work remain at odds with one another, producing independent estimates that completely contradict one another.  In September 2010, the Institute for Health Metrics and Evaluation and the UN published two separate estimates of global maternal mortality.  While the overall estimates for the total number of deaths were very similar, the estimates concerning the causes of death were significantly different.(24) While it is difficult to ascertain which of the two figures is more reliable, it can be reasonably deduced that both estimates cannot be correct.  Juxtaposing the contradictory estimates prompted confusion among governments, professors, and public health officials, as they realized that estimates are not necessarily as accurate as is often assumed.(25)

The Proper Use of Global Health Estimates

Global health estimates have the potential to be a powerful tool in shaping international health priorities. Generating reliable estimates for the future is contingent on decreasing our reliance on complex statistical models by improving data collection in middle- and low-income countries. Reducing the complexity of global health estimates is a necessary step in adhering to sound estimate-generating methodology, which emphasizes the importance of transparency when formulating estimates.(26) According to Peter Byass, a researcher from Umea University in Sweden, increasing methodological complexity of estimates “can lead to an “Emperor’s New Clothes” syndrome in which only the cognoscenti truly understand the underlying basis of complex estimates, while the vast majority may be reluctant to admit that the detail is beyond their comprehension.”(27) When estimates become this intricate, transparency is lost, and the process of peer review is severely jeopardized. While improving transparency is one of the most widely agreed upon changes that should be incorporated into the process of generating global health estimates, experts also outline a number of other critical changes that would likely enhance the estimate reliability.  Additional modifications should include an increase in individual country involvement in data collection and estimate generation, as well as the establishment of a mandatory review process to evaluate the accuracy of estimates conducted by independent expert groups.(28) It is hoped that by involving individual countries, independent expert groups or academic institutions, and UN agencies, more balanced estimates will result.(29)

When searching for estimates to use as part of a publication or program development, it is critical to select figures generated using the principles explained above. Ensuring that an estimate is transparent and has withstood rigorous peer review is essential for producing reliable publications or predictions. It is also critically important to present the figure or statistic as an estimate, and not as a fact. Global health estimates that are generated and presented in this way then have the potential to make significant and lasting contributions to global health research, policy, and reporting. In fact, it is extremely important that the information harnessed via these estimates are reported: “Unless health information is disseminated to the public, the scientific and public health community, and decision makers through multiple channels, including the media, scientific journals, and other documents, it more often than not remains unused in statistical abstracts or spreadsheets in health ministries.”(30)  It is critically important that as a global community, we work not only to improve the overall quality of global health estimates, but also strive towards using these estimates responsibly and respectfully.

Footnotes

(1) Byass, P. (2010). The imperfect world of global health estimates. PLoS medicine, 7(11).

(2) Boerma, J. T., Mathers, C., & Abou-Zahr, C. (2010). WHO and global health monitoring: the way forward. PLoS medicine, 7(11).

(3) Sankoh, O. (2010). Global health estimates: stronger collaboration needed with low-and middle-income countries. PLoS medicine, 7(11).

(4) Graham, W. J., & Adjei, S. (2010). A call for responsible estimation of global health. PLoS medicine, 7(11).

(5) Byass, P. (2010). The imperfect world of global health estimates. PLoS medicine, 7(11).

(6) Murray, C. J., & Lopez, A. D. (2010). Production and analysis of health indicators: the role of academia. PLoS medicine, 7(11).

(7) “Economic Benefits of the Global Polio Eradication Initiative estimated at US$40-50 billion.” The Global Polio Eradication Initiative. (2010). https://www.gatesfoundation.org/Media-Center/Press-Releases/2010/11/Economic-Benefits-of-the-Global-Polio-Eradication-Initiative-Estimated-at-US4050-Billion.

(8) Pruss-Ustun, A., & World Health Organization. (2008). Safer water, better health: costs, benefits and sustainability of interventions to protect and promote health.

(9) “Straight talk with…Christopher Murray.” Nature Medicine. 15.10 (2009): 1104.

(10) Pruss-Ustun, A., & World Health Organization. (2008). Safer water, better health: costs, benefits and sustainability of interventions to protect and promote health.

(11) “Economic Benefits of the Global Polio Eradication Initiative estimated at US$40-50 billion.” The Global Polio Eradication Initiative. (2010). https://www.gatesfoundation.org/Media-Center/Press-Releases/2010/11/Economic-Benefits-of-the-Global-Polio-Eradication-Initiative-Estimated-at-US4050-Billion.

(12) Ibid.

(13) Murray, C. J., & Lopez, A. D. (2010). Production and analysis of health indicators: the role of academia. PLoS medicine, 7(11).

(14) Byass, P. (2010). The imperfect world of global health estimates. PLoS medicine, 7(11).

(15) AbouZahr, C., & Boerma, T. (2005). Health information systems: the foundations of public health. Bulletin of the World Health Organization, 83, 578-583.

(16) Byass, P. (2010). The imperfect world of global health estimates. PLoS medicine, 7(11).

(17) Murray, C. J., Lopez, A. D., & Wibulpolprasert, S. (2004). Monitoring global health: time for new solutions. Bmj, 329(7474), 1096-1100.

(18) Larson, C., & Mercer, A. (2004). Global health indicators: an overview. Cmaj, 171(10), 1199-1200.

(19) Murray, C. J., & Lopez, A. D. (2010). Production and analysis of health indicators: the role of academia. PLoS medicine, 7(11).

(20) Ibid.

(21) Byass, P. (2010). The imperfect world of global health estimates. PLoS medicine, 7(11).

(22) Murray, C. J., & Lopez, A. D. (2010). Production and analysis of health indicators: the role of academia. PLoS medicine, 7(11).

(23) “Straight talk with…Christopher Murray.” Nature Medicine. 15.10 (2009): 1104.

(24) Byass, P. (2010). The imperfect world of global health estimates. PLoS medicine, 7(11).

(25) Graham, W. J., & Adjei, S. (2010). A call for responsible estimation of global health. PLoS medicine, 7(11).

(26) Byass, P. (2010). The imperfect world of global health estimates. PLoS medicine, 7(11).

(27) Ibid.

(28) Boerma, J. T., Mathers, C., & Abou-Zahr, C. (2010). WHO and global health monitoring: the way forward. PLoS medicine, 7(11).

(29) Murray, C. J., & Lopez, A. D. (2010). Production and analysis of health indicators: the role of academia. PLoS medicine, 7(11).

(30) Murray, C. J., Lopez, A. D., & Wibulpolprasert, S. (2004). Monitoring global health: time for new solutions. Bmj, 329(7474), 1096-1100.