STATS ARTICLES 2010
The unbearable lightness of health science reporting
Rebecca Goldin, Ph.D, April 16, 2010
Comparisons between Italian and American health science reporting show itís la dolce media on both sides of the Atlantic.
If a daily dose of journalism is essential for a healthy democracy, it may have a less-than-salutary effect on actual public health. This is the conclusion raised by two projects which set out to determine the quality of information in health science reporting.
In “The Unbearable Lightness of Health Science Reporting: A Week Examining Italian Print Media,” published in the online journal PLoS One, Three independent doctors examined 146 printed health science articles, each one aiming to improve reader’s knowledge of health. They evaluated the articles according to an assortment of parameters, including benefits and costs, associated risks, sources of information, disclosure of conflicts of interest and balance. Balance included whether there were exaggerated or incorrect claims.
Their conclusion? “These findings raise again the fundamental issue whether popular media is detrimental rather than useful to public health.” The researchers found undisclosed costs and risks, undisclosed conflicts of interest, and exaggerated claims. Benefits were exaggerated and risks underplayed. Reports on new medical approaches were considered unbalanced almost nine times as often as other kinds of health-sciences articles.
Of interest to discussions in the U.S. about health care reform is that among articles discussing new medical approaches, costs were mentioned in only 12 percent of stories, though most of these articles discussed costly drugs already on the market. Risks were also reported in only 12 percent of the articles. Benefits, in contrast, were mentioned in all the articles, typically reported in qualitative (rather than quantitative) terms. A third of the time they were quantified, but always in relative, rather than absolute, terms. But even a high relative benefit does not mean that many people overall will benefit, since the population with the problem may be very small. With relative benefit alone, patients and physicians cannot consider the full benefits and costs of a particular treatment option.
Sourcing was also a problem. Only three percent of the articles referenced books; the dominant sources (59 percent) were, instead, interviews with “expert opinion leaders,” defined as professors, researchers, head physicians, representatives of patients' associations or experts of companies. The finding here: Only six percent of the articles cited more than one source.
The problem with one-source stories in health-science communication is that they are vulnerable to conflicts of interest. And this was born out in the data analysis: Though only nine percent of the articles disclosed a financial conflict of interest, the researchers easily found similar conflicts in an additional 10 percent of stories, meaning that less than half of known conflicts were reported. They note that there may have been additional conflicts that were not straight-forward to ferret out.
Unbalanced reporting occurred in 18 percent of the articles under review; however, among those concerning new treatments, tests or products, this jumped to 45 percent. As the researchers note, “In the majority of the unbalanced stories, exaggerated or incorrect claims aimed at or had the effect to favor a new treatment, procedure, test or product.”
How does this compare to the United States? The organization HealthNewsReview.org has been engaged in an similar ongoing its evaluation of health coverage based on explicit ratings criteria. In its latest published report (also published in PLoS Medicine) on the state of U.S. health reporting, over 500 health news stories were analyzed over 22 months, and between 62 percent and 77 percent of stories “failed to adequately address costs, harms, benefits, the quality of the evidence, and the existence of other options when covering health care products and procedures.”
In contrast to Italy, the U.S. media failed to cover benefits as much or more than it did harm: only 28 percent of articles were considered to have “adequately covered benefits,” while 33 percent “adequately covered harms.” But if American journalism is more pessimistic than Italian, it boils down to the same message: the media is doing a poor job at reporting on health sciences. Perhaps journalists should take a closer look at the HealthNewsReview.org criteria for good medical reporting (see sidebar).