STATS ARTICLES 2008
Acupuncture and Fertility: The Media Screws Up
Rebecca Goldin Ph.D, and Jenna Krall, March 3, 2008
Odds ratios can also be prickly
A study published early February in the British Medical Journal examined the link between acupuncture and successful in vitro fertilization. Editorializing headlines had rewritten the science within a week. The study was covered by news organizations, including Fox News, MSNBC, The Examiner, and The Washington Times, all declaring that “fertility benefits from old remedy.” The message was that scientists have now shown conclusively that acupuncture fosters successful pregnancies.
But embedded in the original article is the caveat that the theory is “far from proven.” In the seven studies conducted for the paper, only three found the effect of acupuncture on IVF patients significant. One of the studies found no relation between IVF success and acupuncture, and the remaining three “found a trend toward benefit.” The authors pooled the seven studies in a meta-analysis to find that the odds ratio – the odds of pregnancy through IVF and acupuncture divided by the odds of pregnancy through IVF without acupuncture – was 1.65. Here the word ‘odds’ is used in the statistical sense, not the layman meaning.
The odds of pregnancy through IVF are given by q/(1-q), where q is the probability that someone gets pregnant through IVF. The odds ratio of getting pregnant with acupuncture compared to getting pregnant without it is given by p/(1-p) divided by q/(1-q), where p is the probability that someone gets pregnant through IVF with acupuncture, and q is that probability without acupuncture. The odds ratio can be useful if it is understood in context. In this case, the study itself notes that:
“the odds ratio significantly overestimates the rate ratio in this context, in which the event (pregnancy) is relatively frequent. In absolute terms, the number needed to treat was 10, suggesting that 10 patients would need to be treated with acupuncture to bring about one additional clinical pregnancy.”
The media reported this figure as if acupuncture increases the success rate of IVF by 65 percent, misunderstanding what is meant by the odds ratio. The actual increase in pregnancy likelihood according to this meta-analysis is about ten percent. This alone makes the results far less impressive than they sound.
Though the news reports of this study acknowledged the lack of conclusive evidence supporting acupuncture, the story received lots of media attention. And editors writing headlines seem to undermine the cautious reporting behind it, while missing the consistent mistake made by the press on the meaning of the odds ratio.