EDITORIALS
Making sense of health statistics
Gerd Gigerenzer*
Hardnig Center for Risk Literacy, Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany
Many physicians, patients, health journalists and politicians do not understand health statistics. Yet we make little effort to educate our children in statistical thinking or our medical students in understanding health statistics. This collective statistical illiteracy has resulted in serious consequences for health.1
I argue that statistical illiteracy is largely caused by non-transparent framing of information. Albeit not always intentional, non-transparency is often a deliberate tactic to manipulate or persuade people. I also argue that there is a simple solution: confusing numbers can be easily understood when represented in a transparent way.2 In this editorial, I give two illustrations: unlike relative risks and 5-year-survival rates, which mislead many people, absolute risks and mortality rates provide transparent tools for risk communication.
Relative versus absolute risks. The British Committee on Safety of Medicines issued a warning in 1995 that the third-generation oral contraceptive pill increased the risk of a thrombosis twofold - that is, by 100%. The news, presented in an emergency meeting to the media, caused great anxiety among women taking the pill, many of whom stopped using it. How big is 100%? The studies on which the warning was based showed that, out of every 7000 women who took the pill of the previous generation, 1 had a thrombosis, and that this number had increased to 2 for women who took the third-generation pill. That is, the relative risk increase was 100%, but the absolute risk increase was 1 in 7000. The pill scare led to an estimated 13 000 (!) additional abortions in the following year in England and Wales.3 The resulting cost increase for the National Health Service has been estimated at about £4-6 million. Had the committee and the media reported the absolute risk increase, it would not have caused such panic.
A similar panic may well happen again, since the public (and many physicians) are not informed about the difference between absolute and relative risks. In a study of 150 gynaecologists, one-third did not understand the meaning of a 25% risk reduction created by mammography screening. Most of them believed that, if all women were screened, 25% or 250 fewer women out of every 1000 would die of breast cancer, although the best evidence-based estimate is about 1 in 1000.2,4 I believe it is an ethical imperative that every doctor and patient understand the difference between absolute and relative risks, to protect patients against unnecessary anxiety and manipulation.
5-year survival rates in screening versus mortality rates. When running for president of the United States of America in 2007, the former New York City mayor, Rudi Giuliani, said in a campaign advertisement: "I had prostate cancer, 5, 6 years ago. My chance of surviving prostate cancer - and thank God I was cured of it - in the United States? 82%. My chance of surviving prostate cancer in England? Only 44% under socialized medicine."5 For Giuliani this meant that he was lucky to be living in New York rather than York, since his chances of surviving appeared to be twice as high. This was big news, but also a big mistake. The prostate cancer mortality rate is basically the same in the USA and the United Kingdom. Most importantly, 5-year survival rates and mortality rates are uncorrelated (r = 0.0) across the 20 most common solid tumours.6 In the context of screening, survival rates are misleading statistics. One reason is the lead-time bias. Imagine two groups of men who all die at age 70 of prostate cancer. The men in the first group do not participate in prostate-specific antigen (PSA) screening, and their cancer is detected from symptoms at age 67; thus, their 5-year survival rate is 0%. The second group undergoes screening, and their cancers are detected at age 60, resulting in a 5-year survival rate of 100%. The second reason is overdiagnosis, the detection of non-progressive cancers - abnormalities that meet the pathological definition of cancer but will never progress to cause symptoms in the patient's lifetime. Non-progressive cancers inflate survival rates.
The difference between the survival rates in the USA and the United Kingdom are largely due to the widespread use of PSA screening in the USA, despite lack of evidence that it saves lives, whereas in the United Kingdom, PSA tests are not routinely used. Yet this does not mean that British politicians always have a better understanding of health statistics. When the British Office for National Statistics reported on cancer-survival trends that 5-year survival for colon cancer was 60% in the USA compared to 35% in the United Kingdom, experts dubbed this finding "disgraceful" and called for the government to double its spending on cancer treatment. Prime Minister at the time, Tony Blair, set a target to increase survival rates by 20% over the next 10 years, saying: "We don't match other countries in its prevention, diagnosis and treatment."7 Despite the differences in survival rates, the mortality rate for colon cancer in the United Kingdom is about the same as in the USA.
There is more to transparent risk communication than I can contribute here, detailed in other useful literature.1,2,8 The challenge is to recognize that much confusion about medical statistics is due to non-transparent framing - the rule rather than the exception in health care - and to fund programmes that implement transparency in journals, health pamphlets and doctor-patient interactions. Last but not least, we should begin teaching the next generation how to make sense of health statistics.
References
Available at: http://www.who.int/bulletin/volumes/87/8/09-069872/en/index.html
References
1. Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. Helping doctors and patients to make sense of health statistics. Psychol Sci Public Interest 2007;8:53-96. doi:10.1111/j.1539-6053.2008.00033.x
2. Gigerenzer G. Calculated risks: how to know when numbers deceive you. New York, NY: Simon & Schuster;2002. British version: Reckoning with risk: learning to live with uncertainty. London: Penguin; 2002.
3. Furedi A. The public health implications of the 1995 'pill scare'. Hum Reprod Update 1999;5:621-6. PMID:10652971 doi:10.1093/humupd/5.6.621
4. Covey J. A meta-analysis of the effects of presenting treatment benefits in different formats. Med Decis Making 2007;27:638-54. PMID:17873250 doi:10.1177/0272989X07306783
5. Dobbs M. Rudy wrong on cancer survival chances. Washington Post, 30 October 2007.
6. Welch HG, Schwartz LM, Woloshin S. Are increasing 5-year survival rates evidence of success against cancer? JAMA 2000;283:2975-8. PMID:10865276 doi:10.1001/jama.283.22.2975
7. Steimle S. UK's Tony Blair announces crusade to fight cancer. J Natl Cancer Inst 1999;91:1184-5. PMID:10413410 doi:10.1093/jnci/91.14.1184
8. Woloshin S, Schwartz LM, Welch HG. Know your chances: understanding health statistics. Berkeley, CA: University of California Press; 2008.
* Correspondence to Gerd Gigerenzer (e-mail: gigerenzer@mpib-berlin.mpg.de).