[soundcloud_ultimate track=https://soundcloud.com/adamrodman/the-number-needed-to-treat]
In this episode of Bedside Rounds, we discuss how risks and benefits are communicated by scientists and physicians, and why those numbers you see in advertisements and newspapers might not be the clearest way to express risk.
I’m looking at a newspaper ad from 2009 for Lipitor right now (which will be linked on the website). It has a picture of a futuristic-looking heart in the background, with a cardiganned-Dr. Robert Jarvik, “inventor of the Jarvik Artificial Heart and Lipitor user,” which proclaims “Lipitor reduces risk of heart attack by 36%”. The 36 percent is incredibly large and highlighted, by the way. And why wouldn’t it be? 36% is an awesome number. If I knew I had a 36% chance to win the lottery, damn right I’d go and buy the lottery ticket.
Dr. Jarvik, since you asked, and I didn’t know this, is the husband of the Ask Marilyn columnist in those Parade magazine tabloids that come with your newspaper, and he did in fact pioneer the artificial heart, which is pretty cool. But he came under fire for his role in those ads, including the fact that he is not licensed to practice medicine, and that he used a body double for rowing scenes (he apparently doesn’t row). But the ethics of direct to consumer advertising will have to wait for ANOTHER podcast. But I’m sure you can imagine how I feel…
Examples aren’t limited to drugs. How about cellphones and cancer? Certainly you’ve seen the study that says that heavy cellphone use is associated with a 40% increase of a type of brain cancer called a glioma. No wonder this was front page news for a lot of news organizations. I use my cellphone every day! If say, drinking coffee increased by risk of getting cancer, I’d probably stop that, right? 40% is crazy!
And mammograms! A big meta-analysis, which compared all the randomized-controlled trials on the subject, showed a decrease in breast cancer death of 15%. With numbers like that, you’d be crazy not to call your local breast imaging center and make an appointment.
The three studies that are being quoted, by the way, are ASCOT-LLA, an RCT, for lipitor, the Interphone trial, which was a case-control trial, for cell phones and cancer, and the 2011 Cochrane meta-analysis of RCTs for mammograms. So you can probably see where this is going. And yes, it’s going the take some biostatistics.
The point of this podcast is not to critique any of these trials. All the numbers that I’ve quoted are real, at least in so far as the trials were well-controlled. Pfizer was not lying when it reported that its trial showed Lipitor decreased the risk of heart attacks by 36%. But the ad is reporting the relative risk reduction, not the absolute risk reduction. In the ASCOT-LLA study (why you would name a trial on an antiquated piece of neckwear last made popular by Scooby Doo is beyond me), the risk of having a cardiovascular event was 2.7% in the group that got a placebo, and 1.7 in the group that got atorvastatin (aka Lipitor). That means that the ABSOLUTE risk reduction is 2.7-1.7% or 1%. So the marketing people at Pfizer should probably rework that ad with Dr. Jarvik to say something like, “Lipitor decreases your risk of a heart attack by 1%”, with that 1% big, bold, and red, right?
Reporting the relative risk reduction instead of the absolute risk reduction happens a lot in health reporting, and in advertising, both to doctors and the type of ads that you see on TV. And you can understand why. Big numbers are a lot sexier than small numbers. And one might cynically think that they get the study more attention — whether in eyeballs for a news agency, or in sales for a drug company.
The absolute risk reduction in mortality, by the way, for mammograms was 0.05% in the Cochrane review, which is a little less attention getting than a 15% decrease in death. For gliomas and cellphone use, I didn’t look at the subgroup analysis for Interphone, but the incidence of glioma is very low — about 0.02% in 2008 in the U.S., so a 40% relative risk reduction is going to be a very, very tiny absolute risk reduction.
But I’d argue that absolute risk reductions, while they’re certainly more honest, have their own share of problems when communicating risk to our patients or to the public. How does someone wrap their head around the idea that a mammogram will decrease their risk of death of cancer by half a percentage point? We just aren’t primed to think in terms of risk — I know I’m not, and I run risk calculators with my patients daily.
Which is why I like to use a concept called the number needed to treat to help explain risks to my patients (and myself) in a much more intuitive way. Very simply, the NNT is the number of people who would need a therapy (or need to be screened, in which case it is called the number needed to screen) in order to prevent an outcome. So in the case of the ASCOT trial above, the NNT is the number of people I’d need to treat with atorvastatin (aka Lipitor) to prevent one heart attack. The number needed to screen for mammograms is the number of people who would need to get mammographic surveillance to prevent one death from breath cancer.
And yes, more math is going to be involved here. But the formula is actually pretty easy. The NNT is the inverse of the absolute risk reduction. So an easy example, for the ASCOT trial, the absolute risk reduction was 1%. 1 divided by 1% is 100 — 100 people would need to be treated with lipitor over the study period to prevent one heart attack. And for mammograms, the number needed to screen would be 1 divided by 0.05%, which is 2000. So 2000 women needed mammographic surveillance to prevent a death from breast cancer. I think the NNT — and its corollary for side effects, the number needed to harm — is a much more intuitive way to think about risk. But the kicker? For randomized control trials don’t report it. They often don’t even report the absolute risk reduction.
That’s right. If I want to know the number needed to treat for, say, the latest hepatitis C drug sofosbuvir, I have to make a 2×2 table, pull out a calculator, calculate the absolute risk reduction, then calculate the NNT. And that’s not just because I don’t read enough — and I don’t read enough. A 2002 JAMA article looked at 359 articles on RCTs in the Annals of Internal Medicine, BMJ, JAMA, The Lancet, and the New England Journal. Of those 359 articles in the best of the best journals, only 18 often published the ARR, and only 8 published the NNT.
Now, that being said there are absolutely problems with the number needed to treat. Professor Jane Hutton, a statistician at the University of Warwick in the UK, has written an interesting take-down of the statistic. Her major complaints — there is no accounting for precision — that is, no confidence intervals on an NNT, and the NNT cannot account for a trial in which there is no difference between the two groups. She also asserts that the NNT cannot be used as a summary statistic for meta-analyses, which is exactly what I did with the mammogram study above. The reasoning gets a little complex for me, because, damnit Jim, I’m a doctor, not a statistician. She concludes, “Good RCTs and meta-analyses should not be down-graded by using NNT when ARR is reliable and comprehensible.” Though mind you, as the previous study showed, ARR isn’t exactly frequently published either.
And it’s not just me who thinks there should be more reporting of the NNT (and the ARR). A standards group of scientists and editors called the Consolidated Standard of Reporting Trials (CONSORT) issued a statement in 2001 (updated in 2010, and available online at consort-statement.org) that recommends reporting both absolute AND relative effect sizes for binary outcomes, and in particular stated that number needed to treat and NNH were “helpful”.
So if studies don’t publish NNTs, or even ARRs, how are we supposed to easily calculate numbers needed to treat, ideally without extensive pubmed searches and calculators? Fortunately, it’s already been done for us. TheNNT.com is a website started by several emergency medicine residents that looks at high-quality meta-analyses of RCTs (usually Cochrane reviews, but always a study that has been vetted by the ACP Journal Club) and calculates out the NNT and the NNH — or the relative risk reduction, if you so desire.
Let’s check out aspirin to prevent a first heart attack or stroke, which we call primary prevention, only because a daily aspirin is something that a lot of patients do. The source is a 2009 meta-analysis in the Lancet. We find that the NNT to prevent any cardiovascular problem in 1,667; these were all non-fatal problems, as no deaths were prevented. The number needed to harm in a major bleeding events requiring hospitalization and transfusion was 3333. Now there’s plenty to quibble about in such an analysis — the website rates aspirin for primary prevention as “red” meaning not recommended based on this analysis, and most professional societies, including the USPSTF recommend aspirin for higher risk groups.
The website theNNT isn’t perfect, and it brings to mind some of the criticisms that Professor Hutton, the statistician had raised. But if authors aren’t publishing their NNTs — or even their ARR — I’m glad that someone else is doing it. Because Pfizer — and other drug companies — isn’t in the business of providing the clearest picture possible of the benefits of their medications. They’re in the business of selling drugs. But as physicians and patients — and that’s basically everyone, since virtually everyone will experience a medical intervention of some sort at some point in their life — our business should be having the clearest picture of the risks and benefit of different interventions. And keeping the ARR and the NNT in mind can help us do that.
That’s it for the show. Thanks so much for listening! We had some great feedback from last week’s episode Beachside Rounds. In particular, my friend Dr. Daughty pointed out that you can in fact examine the lymphatic system from a distance, and pointed out the neck-bulging Burkitt’s lymphoma. Another listener liked to look for exophthalmos, or bulging eyes, seen classically in the hyperthyroid state Grave’s disease.
As for our sources we have:
Hutton, Misleading Statistics, Pharmaceutical Medicine, June 2010
Nuovo et al, Reporting Number Needed to Treat and Absolute Risk Reduction in Randomized Controlled Trials, JAMA, 2002;287(21):2813-2814.
www.consort-statement.org
The three trials were mentioned were ASCOT-LLA, Interphone, and the Cochrane review called Screening for Breast Cancer with Mammography.