STATS ARTICLES 2010
The flu: Itís about variance
Rebecca Goldin Ph.D, September 3, 2010
If you averaged the highs and lows of a rollercoaster, it wouldnít be much of a thrill ride; same with the threat of flu.
As summer draws to a close, news organizations are already looking to this year’s flu season, perhaps wary of the way Swine flu fears fizzled. Now the the Centers for Disease Control (CDC) has given reporters, it seems, another reason to be suspicious of flu as a threat to public health.
The CDC has acknowledged that estimates of deaths due the flu have been misleading. According to CDC, there are two ways in which the numbers are misleading. First, the “average number of deaths per year” during the period 1976-2007 is not 36,000, as the CDC has often claimed. Second, the notion of average is itself misleading.
The reason that the average is misleading is that the variance is so high. The variance measures how far the data fall from the average. The CDC noted that numbers of flu deaths have been as low as 3,300 and as high as almost 50,000 in various years. For any particular year, the actual number of deaths could be vastly different from the average number of deaths. They could be so different, in fact, that the average number itself does not communicate the scope of the issue.
In light of the widespread use of the average number of deaths (which, news organizations noted, were closer to 23,000 than 36,000), the CDC explicitly encourages news organizations to use vaguer numbers, such as “tens of thousands” rather than referring to 23,000. The true news about the flu is not the average number of deaths, but rather that the number of deaths varies widely from one year to the next, and could in fact be much higher in one year compared to another. By reporting averages, the CDC and news media suggest that the actual flu deaths are close to those averages.
Does this matter? For many, the answer is yes. If an elderly person is told that “on average” about 1.7 in 10,000 people suffered influenza-associated deaths, she may react differently than if she hears that this number has been as low as .24 deaths in 10,000 (about one in 40,000) or as high as 3.67 deaths in 10,000 (about one in 2,700 people). The risk that it could be so high could prompt some to get flu shots, while the fact that it might be very low may console others. It also points to how difficult it is to predict the outcomes of disease year to year and why public health officials are concerned about bad flu years.
Unfortunately, news organizations added their own misleading picture of how the numbers were misleading, at least in the headlines: The LA Times claimed that “Figures on flu deaths are misleading, usually too high, CDC says,” which suggests that the misleading aspect is the averages (the figure of 36,000 instead of 23,000) rather than the variation of numbers. The New York Times didn’t do much better with “Estimate Lowered of Typical Flu Toll.” While technically true, it avoids noting that reporting the estimates as strict numbers is not the whole story. Both stories did end up discussing variation. Reuters led with the headline, “CDC backs away from decades-old flu death estimate.” Business Week noted that “CDC Revises Death Estimates From Flu” but continued with the teaser, “Mortality ranged from an annual low of 3,300 to a high of 49,000 during last 30 years.”
Of all news sources, we found only one with a descriptively accurate headline - Infection Control Today (not much of a shock there) with “New Report Shows Variability in Annual Rate of Flu-Related Deaths.”
Perhaps the mainstream media should take a fresh look at reporting averages and variation; while the average media account wasn’t all that bad, there was practically no variation at all on coverage of the CDC’s flu announcement.