STATS ARTICLES 2011
Alcohol and Accidents: Can One Drink Kill?
Dr. S. Robert Lichter, July 14, 2011
A recently released study has attracted media attention by suggesting that no level of alcohol consumption is too low to produce serious negative effects behind the wheel; however, a closer look at the findings shows they may not quite be so newsworthy.
A recently released study has attracted media attention by suggesting that no level of alcohol consumption is too low to produce serious negative effects on drivers who get into accidents. The findings received extensive coverage from prominent news organizations, such as WebMD, ABC News, and TIME which proclaimed “Driving While Buzzed: No Amount of Alcohol Is Safe Behind the Wheel.”
The article, by sociologists David Phillips and Kimberly Brewer, appears in the most recent issue of the journal Addiction.1 The authors find that any amount of alcohol consumed by drivers increases the severity of injuries in automobile accidents. On the basis of this finding, they argue that the legal limits of blood alcohol concentration for drivers should be lowered below the current level: “The severity of life-threatening motor vehicle accidents increases significantly at blood alcohol concentrations far lower than the current US limit of .08%.”2
Their data comes from the Fatality Analysis Reporting System (FARS), a data set which has the advantages of including all fatal auto accidents in the US and measuring blood alcohol concentration (BAC) in increments of only 0.01%. As they note, it has the disadvantage of including only fatal accidents. So any variation in injuries can be determined only for the most severe accidents, i.e., those in which someone is killed.
In measuring the severity of accidents, FARS breaks down injuries into five categories: (a) no injury, (b) possible injury, (c) non-incapacitating injury, (d) incapacitating injury, and (e) fatal injury. The authors define severity as the ratio between the most severe accidents (d and e) and the least severe (a and b). They leave out (c) – “non-incapacitating injury” – because they consider it “potentially ambiguous.”3 Thus, their “severity ratio” equals (d+e) / (a+b). For example, if there were twice as many severe as non-severe accidents, the severity ratio would equal two.
Unfortunately, the decision to leave out the middle category in the severity ratio might affect the results in ways that we cannot determine. Imagine a presidential approval poll that examined the ratio between those who strongly approved of the president and those who strongly disapproved, while leaving out those who expressed moderate approval or disapproval. Our conclusions about the president’s popularity might look quite different according to whether those intermediate groups were included in the comparison.
In addition, by leaving out one of the five levels of injury, severity, this procedure makes it more difficult to determine whether a linear relationship exists between accident severity and BAC level. In any case, however, the authors do not perform the kind of data analysis necessary to establish such a relationship. They are interested not in the overall relationship between these two variables, but only in whether accident severity increases at the lowest measured levels of BAC. They test for this by calculating the statistical significance of the difference in accident severity at the .00% and .01% levels of drivers’ BAC.
As every student of statistics quickly learns, statistical significance is not the same as substantive significance. A relationship can be statistically significant but substantively unimportant. The latter is often addressed by going beyond the question of whether an association exists to ask how strong the relationship is. But there is an even more basic problem in this case -- statistical significance tests are inappropriate for this data analysis. They can determine whether two values derived from a random sample are different enough that they cannot be attributed to chance (sampling error). But the FARS data are not a sample of some larger data set; they represent the entire data set of fatal auto accidents. The authors note this problem but choose to use this approach anyway, following “official recommendations and our previous practice.”4
The paper’s principal statistical analysis consists of comparing the injury severity ratios of accidents involving drivers with different BAC levels. The key finding is that any driver’s BAC level above zero increases the severity ratio of injuries suffered in fatal accidents. In effect, the authors conclude, there is no safe level of alcohol consumption for drivers: “Accident severity increases significantly even when the driver is merely ‘buzzed.’”5 Moreover, this finding holds true “throughout the US, throughout the study period, and for both single- and multiple-vehicle crashes.”6
As their data analysis shows, however, it does not hold true for everyone involved in accidents. The severity of injuries increases appreciably for drivers, but not for their passengers or for pedestrians or others outside the driver’s vehicle. One might expect that, if more severe injuries are experienced by “buzzed” drivers in accidents, this would also show up in the seriousness of injuries to other people involved in the same accidents.
Another limitation is more clearcut. Although the authors state twice that their findings “hold throughout the US,”7 this is contradicted by their published data. An increase in BAC level from .00% to .01% produces a statistically significant increase in injury severity in the South, the Midwest, and (to a lesser extent) the Northeast. But the opposite holds true in the West, where a higher BAC level produces a significantly lower injury severity level. This is true for both tables in which this statistic appears – injuries inside the driver’s vehicle (Table 1, p=.0025) and drivers’ injuries only (Table 2, p=.0016). The source of this anomaly isn’t clear. But when three out of four categories move in the hypothesized direction and the fourth category moves in the opposite direction, something is going on in the data that needs to be addressed.
The most important limitation of the study stems from the authors’ decision to focus their data analysis on the differences between .00% and .01% blood alcohol levels, although they include bar graphs (Figures 1 and 2) tracking the ratio of serious to non-serious injuries against blood alcohol levels in .01% increments all the way up to .25%. The article does not include the actual data points, but by eyeballing the key graphs of injuries inside the driver’s vehicle and driver injuries alone (Figures 1A 8 and 2A 9), we can see that the relative seriousness of injuries rises from .00% to .01% of blood alcohol concentrations and rises again, to a lesser extent, from .01% to .02% BAC.
However, the graphs also show that the relative seriousness of injuries drops when BAC rises to .03%, and it remains stable at .04%, before rising at .05% to about the same level of severity found at .02%. Following the authors’ policy inferences from their findings, one might advise drivers to avoid that first drink, but if they are already up to .02% BAC, they should take another drink to bring them up to .03% or .04% BAC, since that would reduce their likelihood of serious injury. Of course, no one would seriously make this suggestion.. But this shows the danger of prescribing policy based on a data analysis that is confined to the last two data points at one end of the tail of a statistical curve.
Where does this critique leave us in describing the relationship between drinking and the seriousness of injuries in accidents? Despite the absence of a formal data analysis to this effect, the graphs in the article clearly show that gradual increases in blood alcohol levels up to .25% are positively related to the relative seriousness of injuries to drivers involved in fatal accidents. Presumably many of these fatalities are the drivers themselves, since the relationship does not appear to hold for passengers, pedestrians, or the drivers of the other vehicle(s) involved in these accidents.
However, a number of caveats are necessary in order to accept the conclusion the authors present as their key finding -- that this relationship exists for the first incremental increase in BAC, from .00% to .01%. As they put it, “S [serious injury] increases significantly even if the driver is merely ‘buzzed,’ i.e., has any detectable BAC” [emphasis in original, p. 2].
Based on our critique, we might amend the article’s conclusion as follows:
In fatal automobile accidents, an increase from .00% to .01% or .02% in a driver’s blood alcohol level significantly increases the probability that injuries to the driver, but not to anyone else involved, will be relatively more serious, except in the West, where the driver’s injuries will be relatively less serious. However, a further increase from of blood alcohol level from .02% to .03% or .04% may reduce the probability of the driver suffering relatively serious injuries. In addition, these calculations exclude the effects of moderately serious injuries. Finally, the use of statistical significance is technically inappropriate for testing this hypothesized relationship, since the data do not constitute a sample.
This revised conclusion has the advantage of being empirically accurate and methodologically defensible. Alas, it has the disadvantage of not being quite so newsworthy.
1. Phillips, D. P. and Brewer, K. M. (2011), The relationship between serious injury and blood alcohol concentration (BAC) in fatal motor vehicle accidents: BAC = 0.01% is associated with significantly more dangerous accidents than BAC = 0.00%. Addiction. doi: 10.1111/j.1360-0443.2011.03472.x
2. p. 1
3. p. 2
4. p. 2
5. p. 1
7. pp.2, 5
8. p. 3
9. p. 4