December 2000 · Vol. 49, No. 12
An Evidence-Based Approach to Medical Care Raises Uncomfortable QuestionsDavid C. Slawson, MD
The analysis of the screening mammography data in The Lancet1 and our subsequent Patient-Oriented Evidence that Matters (POEMs) review2 have accomplished an important goal: They got people talking about the benefits versus the risks of this common medical intervention. The practice of evidence-based medicine always begins with a question. The editorial by Dr Woolf3 shows that a POEM review can have the effect of generating questions in the minds of readers.
The analysis of the data supporting screening mammography by Gøtzsche and Olsen1 focused on 2 limitations of the available research. The first is the appearance of data loss in the trials indicated by differing reports of the number of patients in several of the trials. The databases in these studies are very large (more than 500,000 women have been studied) making them unwieldy and subject to mistakes.
The second limitation found in these trials is the one that causes the most concern: the inadequate concealment of allocation assignment. Researchers enrolling a patient into the trial knew or could have known whether a woman would be in the screened or unscreened group and could have chosen not to enroll a particular patient or type of patient. According to Gøtzsche and Olsen, this knowledge resulted in an imbalance within the 2 groups.
The importance of study design
Along with randomization and blinding, concealed allocation is a way to keep researchers from succumbing to those human traits that get in the way of the scientific method. The Cochrane Collaboration4 considers concealed allocation more important than the method of randomization for preventing the introduction of bias into a study, and others5 consider it essential for the prevention of selection bias.4 This lack of concealed allocation is a serious flaw—not a random error for which a meta-analysis of multiple studies can compensate—that can be responsible for the same bias occurring over and over in different studies, as was demonstrated in this analysis. Studies without concealed allocation may markedly overestimate a benefit of treatment.5-8
The issues of study design seem to be technical nitpicking until one realizes that the difference in deaths of women screened for breast cancer compared with those who were not screened was only 65 out of a population of almost 0.5 million women. Although the screening may have benefited many women, this benefit has to be weighed against the lack of benefit or actual harm that might have been experienced by others. Ewart9 has discussed the ethics of screening programs such as this one and concluded that screening tests must be shown to be beneficial beyond a reasonable doubt before widespread use. The article by Gøtzsche and Olsen and the POEM review suggest that this criterion is not met with our current research.
The poems process
The primary goal of the POEMs section in JFP is to present relevant and accurate new research findings in a way that makes them accessible to the busy practicing clinician.* The studies are rigorously evaluated in a redundant process by the primary writer of the review, an editor, members of our editorial board at the University of Missouri, and a supervising editor. In recognition of the fact that the typical clinician has fewer than 30 minutes per week to devote to foraging for new information, each review is limited to 700 words or less. In this short space we try to achieve a balance of information that is clear and concise yet points out the limitations of the research. Reviews that painstakingly outline the complete critical appraisal for each article would be complete but essentially unreadable for all but the most fanatical reader.
The best available evidence
As stated by Dr Woolf, clinical practice guidelines that are based on and linked to clinical evidence are excellent tools for directing practice decisions. However, guidelines often take years to produce, may be influenced by politics more than an explicit evaluation of the best evidence,10,11 and are not available for many clinical problems. Physicians must act now, using the best available evidence, and they must know today how to integrate a new research finding into their clinical practices. Some articles should change practice. We focus on articles that use patient-oriented outcomes, because we feel that these are much less susceptible to the ping-pong seen when the focus is on disease-oriented outcomes.
As our technological armamentarium increases in size and invasiveness, it is important that we remember the dictum primum non nocere (first do no harm). These days we can do a lot of harm if we are not careful. Absolute certainty is absolutely impossible, and we do not have to wait for that, of course. The results of the reanalysis of the mammography research data suggest no benefit. We evaluated this analysis and reported the results in the POEMs review. It caused several people to ask questions.,12 That is a good thing.
- Gøtzsche P,
Is screening for breast cancer with mammography justifiable? Lancet 2000;355:129–34.
- Wilkerson BF,
Screening mammography may not be effective at any age. J Fam Pract 2000;49:302, 371.
- Woolf SH.
Taking critical appraisal to extremes: the need for balance in the evaluation of evidence. J Fam Pract 2000;49:xxxx–xxxx.
- Cochrane Collaboration. Cochrane reviewers’ handbook 4.1. Updated June 2000. Available at www.cochrane.dk/cochrane/handbook/hbook63_Selection_bias.htm. Accessed September 18, 2000.
- National Health Service Centre for Reviews and Dissemination. Undertaking systematic reviews of research on effectiveness. CRD Report Number 4. 2nd ed. Available at www.york.ac.uk/inst/crd/report4.htm. Accessed September 18, 2000.
- Chalmers TC,
Bias in treatment assignment in controlled clinical trials. N Engl J Med 1983;309:1358–61.
- Schulz KF,
Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408–12.
- Moher D,
Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609–13.
- Ewart R.
Primum non nocere and the screening of evidence: rethinking the ethics of screening. J Am Board Fam Pract 2000;13:188–96.
- Taubes G.
NCI reverses one expert panel, sides with another. Science 1997;276:27–28.
- Berg AO.
Clinical practice guideline panels: personal experience. J Am Board Fam Pract 1996;9:366–70.
- Steiner E.
The effectiveness of screening mammography. J Fam Pract 2000;49:853.
The Journal of Family Practice ©2000 Quadrant HealthCom Inc.