STATS ARTICLES 2005
Are Aleve’s Problems Due to Chance?
January 12, 2005
Media ignores statistical significance in evaluating risk.
Journalism and science are often at odds—reporters want stories that grab readers with surprising and important information, while science, ultimately, seeks to place risks in context. In the Aleve story, this conflict resulted in the reporting of preliminary figures. Even the scientists working with the data do not yet know whether these figures represent actual risks, or whether they are just the result of coincidence.
In any study, there is a possibility that a phenomenon (such as a higher heart attack rate) occurs randomly. This is why scientists must first determine whether the results are “statistically significant” — meaning that the observed differences between the two groups are unlikely to occur by chance, and are therefore correlated to the drug being studied.
An analogy can be made with coin flipping. Someone flipping a coin 50 times might not get exactly 25 head and 25 tails; a result such as 27 heads and 23 tails wouldn’t surprise anyone. On the other hand, it would be very unlikely to obtain 49 heads and one tail if the coin weren’t weighted in some way.
Similarly, the rate heart of attacks in two groups of people, one taking Aleve and the other taking a placebo, may not be evenly distributed, even if the difference was due to chance alone. Unlike the coin-flipping scenario, we do not know in advance in a clinical trial that any difference is caused by randomness. Depending on how many heart attacks are observed and how skewed they are distributed among the two groups, the result may be statistically significant, and unlikely to occur by chance.
Again, “statistically significant” means the result most likely did not occur by chance; “not statistically significant” means there is a good possibility that the result is due to sheer randomness.
According to John Breitner of University of Washington (a lead researcher in the study on Aleve), the final analysis regarding the statistical significance of the heart attacks and strokes among Aleve users (compared to placebo users) has yet to be completed.
Yet the Washington Post headlined “Aleve Ingredient Joins Painkillers Linked to Risks,” the Boston Globe titled their piece “Another Painkiller Tied to Heart Attack Risk,” and the New York Times declared “Study Links a Fourth Painkiller to an Increase in Heart Problems,” all referring to naproxen, the main ingredient of Aleve.
The news media alerted the public to the “risk” before the risk was determined to be real.
In addition, few articles emphasized that the study’s participants were elderly and had taken the drug for three years before the study was halted. Many Aleve users are not in this category—and the elderly face different risks than younger people. For one, they are more prone to heart attack in general, which is why no study looking at heart attacks would compare the rates amongst young people given a drug to those of the elderly given the same drug. The difference between the groups there would be “confounded” by age: since drug or no, older people are more likely than young people to have heart attacks.
Just as troubling, most of the coverage failed to mention that other studies of naproxen (with various doses, over various lengths of time, by various patient types) found that patients taking the drug had a reduced risk of heart attack.
For example, researchers of an article published in the Archives of Internal Medicine in 2002 found that “patients with rheumatoid arthritis and a current prescription for naproxen had a reduced risk of [heart attacks, sudden death and stroke] relative to those with no naproxen prescription in the past year.” And the Journal of American College Cardiology published a study this year demonstrating the heart protective features of naproxen amongst people who do not take aspirin. (“The effects of nonselective non-aspirin non-steroidal anti-inflammatory medications on the risk of nonfatal myocardial infarction and their interaction with aspirin,” March 2004).
Until the data is fully analyzed, the researchers won’t know whether the risk that appears to be associated with Aleve is actually due to the drug and not chance. Consequently, it is highly irresponsible for news organizations to scare the general public by uncritically reporting news about unconfirmed risks, which in this case only applied to a narrow demographic of people.