2009 | 2008 | 2007 | 2006 | 2005 | 2004 | 2003

Your Water Bottle is Not Going to Give You a Heart Attack
Trevor Butterwort and Rebecca Goldin Ph.D, September 18, 2008 (updated September 19)
Some good and some bad reporting on a new study in JAMA linking BPA to common diseases. What did the study really say, and why did the lead author say we shouldn’t read too much into the findings?

The media responded  to a new study on bisphenol A (BPA), published this week in the Journal of the American Medical Association (JAMA), with hundreds of news stories claiming that the chemical had been “linked” to the occurrence of heart disease and diabetes in people. But given the history of alarmist stories in the media over BPA, and the repeated refusal of the Food and Drug Administration in the U.S. and the European Food Safety Agency to place any restrictions on the use of the chemical in polycarbonate bottles and the epoxy resins used to line cans, the question is did the press report this new study’s findings accurately or did they spin the findings to support stronger (and perhaps scarier) conclusions?

Time Magazine reported that the JAMA study could change everyone’s perception of the risks of BPA; but the study’s lead author, David Melzer, told WebMD, “Please don't read too much into this first study; it's limited in various ways.”

What did the JAMA study find?
Melzer and the other authors found an association between higher urinary BPA levels and heart disease and diabetes in a broad sample of people, and they were careful to consider possible “confounders” – known issues that could inadvertently affect the results, like body mass index (BMI) or smoking. In this case, the range of possible confounders they also looked at included income, education level, age, sex, race/ethnicity, education, waist circumference, and urinary creatinine concentration. By controlling the confounders, they were able to exclude the possibility that, for example, the association between higher BPA and diabetes was actually indicative that people with higher BMI have higher BPA levels, and also a higher risk for diabetes.

However, the authors did not control for family history of heart disease or diabetes, both of which are well accepted predictors for these diseases.

The study itself was cross-sectional in design, which means it can only measure correlations and not causations. It is incorrect, as the authors noted, to conclude a causal relationship in their findings. Without more information and other epidemiological and toxicological evidence, the evidence that BPA causes diabetes or heart disease is about as strong as the evidence that diabetes or heart disease leads to higher levels of BPA in the urine.

The JAMA study in depth: Weaknesses
The first thing that should be noted is that this research doesn't support a key principle of the "BPA is dangerous" lobby, which is that the risk from BPA occurs at extremely low doses as well as very high doses, along a U-shaped (non-monotonic) curve. This has been the key argument put forward by Frederick Vom Saal over the past decade, and is discussed in the accompanying editorial. The associations found in the JAMA study were all at low doses, compard to the high doses that have been shown to have adverse effects in laboratory animals. The "high levels" of BPA in urine that had a statistical association with certain diseases were only "high" compared to the other people in the data sample – they were all "low" levels compared to much of the laboratory research on BPA. This research can neither support nor dismiss the hypothesis that the adverse effects of BPA follow a U-shaped curve.

The JAMA study is cross sectional, which means that it is a snapshot it time; by the nature of its design, the observations are correlations and cannot establish causation. Since the worrisome effects of BPA are likely to be chronic (based on long-term exposure), and yet BPA is metabolized so quickly there may be huge variability in BPA exposure levels for any individual over time. The nature of the cross-sectional study is to assume that the urinary BPA – and, implicitly, the BPA exposure level – are constant over time. But a one-time urine sample may not reflect chronic BPA exposure levels, leaving much more room for a statistical fluke to lead us astray.

In other words, the study design is equivalent to placing a nail on a piece of wood without hammering it in. It doesn’t show whether heart disease or diabetes increases relative to increases in urinary BPA over time. This would require a longitudinal study, and it would also require assessing exposures to other chemicals in food that are much higher in estrogenic potency than BPA, such as zearalenone, which is present in cereals and high calorie foods such as beer and baked goods. There are many possible correlations to consider in disentangling the causes of heart disease, and BPA is just one of many contenders.

Another weakness of the result, to continue the hammer analogy, is the “hammer everywhere to hit the nail” effect. If you hammer enough times, you’re likely to get a hit. Since the study was looking at many possibilities (eight different categories of disease, and several blood markers), it is very likely that it would find some statistically significant effect in its sample, even if this effect wouldn’t be seen in the population as a whole.

The mathematics is basic probability. Imagine that we conduct an experiment in which 15 people toss a coin 10 times each. There is about a five percent chance that any particular individual will get two or fewer tails. However, there is a 57 percent chance that someone will get two or fewer tails. Now suppose that BPA has no effect on any of fifteen health conditions or blood levels, but that, for each of these diseases, by chance alone there is a five percent chance we’ll see an association. Seeing an association that isn’t there seems unlikely (less than five percent); but if we test for all 15 possible health conditions, there is a high chance (over 50 percent) we would see an association with at least one of the markers, even though it is not present in the whole population.

Finally, as discussed above, the causal factors for the observations in this study may simply be correlated with BPA exposure. The reasons that some people have more cardiovascular disease may also be reasons that they have higher BPA levels, including many possible factors. This study does not allow us to tease out the causes from the correlates.

Strengths
All this being said, certain aspects of this study were interesting and provide grounds for further research: If all the diseases showed a correlation with BPA level, then you'd start to wonder whether there are correlated behaviors with BPA exposure that put people at risk. For example, if people who eat more foods that come in plastic are also more likely to sit in front of the TV, an effect due to a sedentary lifestyle could be correlated with BPA.

But the study found that only some of the diseases were associated with higher BPA levels, and diabetes is one of the very diseases that were of concern considering the possible mechanism of BPA harm. The association between high urinary BPA and cardiovascular disease, however, was not an expected outcome.

Similarly, when it comes to causation versus correlation, this study managed to dismiss some reasons for the observed association between BPA, and cardiovascular disease and diabetes. One possible explanation for an observed association is that people who are more exposed to BPA are also more likely to have lifestyles that specifically effect the diseases that BPA was correlated with. As an illustration, perhaps people who have higher urinary BPA levels get their BPA exposure through fast food containers, and these same people would be more likely to have heart disease and diabetes, thanks to the fast food.  The argument against this is that the JAMA study controlled for BMI and waste line, among other factors, and still found the correlation. 

Finally, with regard to the many statistical tests that were performed, one can counter that the p-values were very low (much lower than 0.05), and are unlikely to have occurred just by chance (though it's always possible).

What can we conclude?
So what is the take home message from this study? It’s an interesting study, but it is not more interesting, more significant, or more relevant to public health than the studies on BPA that the media have ignored. And it certainly doesn’t say that your water bottle is going to give you a heart attack.

How did the media do?
Coverage of the study ran the gamut of reporting. There was patent absurdity: “Water bottles that contain Bisphenol A -- also known as BPA -- raise the risk of heart disease and diabetes in people, according to a new study,” claimed ABC’s Honolulu affiliate, The direct statement of causality -- that BPA actually raises the risk -- is a leap of faith from this study, which neither said that BPA raised the risk of heart disease or diabetes, or that water bottles were responsible for people’s exposure to BPA.  Similarly, news-medical.net falsely declared causality by stating “This is the first time BPA has been shown to directly cause harm to humans.”

Then there was inaccurate and incomplete reporting: In claiming that “Many experts already think the writing is on the wall for the chemical,” The Los Angeles Times reported that

“The chemical industry and the FDA have long relied on two large animal studies which showed that high concentrations of the chemical fed to the rodents produced no serious adverse effects.”

In fact, the risk assessments undertaken by the Center for the Evaluation of Risks to Human Reproduction, the European Food Safety Agency, the Japanese government, and NSF International, a non-profit consumer research organization examined many hundreds of studies on BPA, sifting through those which were statistically and methodologically robust before issuing their conclusions. The chemical industry financed two other risk assessments, one conducted by Harvard Center for Risk Analysis, which also evaluated hundreds of studies. All of these risk assessments are in broad agreement that BPA is safe at low doses. It is this body of evidence that the FDA and EPA are drawing on, not simply two studies.

But in a marked change from the kind of hype that has plagued questions of chemical safety, the Associated Press led with the FDA’s defense of BPA’s safety and gave significant space to independent criticisms of the JAMA study’s conclusions, which produced a piece that showed rare balance on this controversial topic.

Two Dartmouth College analysts of medical research said the study, which does not prove that BPA caused the health problems, raised questions but provided no answers about whether the ubiquitous chemical is harmful.

Dr. Lisa Schwartz and Dr. Steven Woloshin of the Dartmouth Institute for Health Policy and Clinical Practice said the study presented no clear information about what might have caused participants’ heart disease and diabetes. The study did not look at exposure to BPA, but only at levels measured in a single urine test. “Measuring who has disease and high BPA levels at a single point in time cannot tell you which comes first,” Dr. Schwartz said.

The researchers did not look for disorders that they thought would be linked to BPA. Instead they did a broad search for diseases that might show an association, generally providing weaker evidence. In particular, there was no prior reason to suspect that heart disease might be linked to BPA. And the researchers used statistical modeling of the health survey population to make their comparisons. For example, fatter people had higher levels of the compound, and the researchers tried to correct for that with statistics.” (via the New York Times).

The danger in overhyping the JAMA study is that it is reductive: the problems of heart disease and diabetes, which have multiple inter-related causes, are presented in a way which seem to connect them to one simple cause, in this case the presence of miniscule amounts of a chemical, which is known to be rapidly and almost entirely excreted from our bodies. It is rare for a news account to go in the direction of the AP and place the emphasis of the news story on what the scientific study doesn’t say. But it is the correct and responsible approach given how scientific data is routinely misinterpreted by the media.

Choosing what to report: The media loves only bad news on BPA
One of the most problematic aspects of the way the media have covered the charge that BPA is a threat to public health is that reporters have systematically failed to report on studies which did not find a risk or risk assessments which found BPA to be safe. For example, over the past year there have been at least three major non-industry funded studies of BPA which are crucial to understanding the state of the science with respect to whether the chemical poses a risk to health.

The first appeared in one of the leading toxicology journals, Toxicological Sciences in December 2007. “Gestational and Lactational Exposure To Ethinyl estradiol, But Not Bisphenol A, Decreases Androgen-dependent Reproductive Organ Weights and Epididymal Sperm Abundance In The Male Long Evans Hooded Rat” was conducted by EPA scientists and found that low dose oral exposures to ethinyl estradiol, a sex hormone used in the contraceptive pill, led to decreased sperm counts and other reproductive effects in rats, while similar low-dose exposures to BPA produced no effects whatsoever.

This was significant because it provided robust confirmation that low dose exposures to chemicals could impair reproductive functioning, but that low-dose exposures to BPA in particular did not show any such effects. These findings challenged a long-standing claim by the “BPA is dangerous” lobby that the chemical caused effects at low doses which could’t be seen at higher doses and thus required more stringent regulation.

The second was a two-year analysis of BPA undertaken by California EPA Toxicologist Calvin Willhite for NSF International, a non-profit international consumer research organization, which was published in the Journal of Toxicology and Environmental Health in February 2008. The goal was to produce an accurate reference dose for oral ingestion, which required systematic evaluation of the body of research on BPA to determine the risks to humans. Willhite’s study failed to find evidence that BPA, in the doses normally ingested by people, posed any clear indication of risk. It was only when BPA was injected directly into the bloodstream of animals, or ingested orally in huge quantities that negative health effects were seen, and both of these kinds of exposure were not relevant to human risk.

The third is the European Food Safety Authority (EFSA)’s two-part update to its 2006 Risk Assessment, originally conducted by 21 independent scientists across the European Union, and which reaffirmed the safety of the BPA to the point where the tolerable daily intake was raised by a factor of five. (In other words, the EFSA found that humans were less at risk than previously thought, and that they could be exposed to more BPA without worry of adverse health consequences.) In July 2008, EFSA responded to concerns in Canada that fetal and infant low dose exposures to BPA could pose a hitherto unrealized threat by noting that Canada was relying on research that was “limited in rigor, consistency and biological plausibility.” They noted that humans metabolize BPA quickly and efficiently compared to rats, and that

“the exposure of a human fetus to free BPA would be negligible due to the maternal capacity for conjugation whereas the fetal rat would be exposed to free BPA from the maternal circulation.”

The second update responded to “some concern” at the U.S. National Toxicology Program about low dose exposures and neurodevelopmental toxicity. EFSA pointed to a report by the European Commission’s Joint Research Centre which concluded that

“due to the low confidence in the reliability of the developmental neurotoxicity studies and the lack of consistency in the results of behavioural testing, no conclusions can be drawn from these studies.”

All of these studies were largely ignored by the media, even though they rebutted in the strongest terms the claims of risk that some smaller studies and the media were reporting.

One could speculate endlessly as to why this has been the case: Perhaps the leading scientific proponents of the theory that BPA poses a risk, notably Frederick Vom Saal, are better at summarizing their point of view in a way that fits more easily with media narratives about risk? Perhaps he who shouts the loudest in the scientific marketplace drives the news?

It could be that the media are especially suspicious of any “synthetic” chemical given the public’s general suspicion that synthetic chemicals are more dangerous than naturally occurring ones (a gut association unsupported by science but deftly explained in Daniel Gardiner’s new book “The Science of Fear”).

It is possible that the allegations made by environmental activist groups (who back the “BPA is dangerous hypothesis” with religious certitude) that industry and regulatory bodies are engaged in a cover-up of evidence against the chemical manages to hook reporters with the idea of a Big Tobacco style cover-up.

Many reporters appear to be unaware that the various independent risk assessments of the evidence on BPA have evaluated many hundreds of studies on BPA and they have been quite explicit in their final reports about why they have selected some studies and rejected much of the independent “BPA is dangerous” research.  Indeed, the reasons are banal – lack of statistical power, lack of multigenerational testing and so on – but good, in the sense that there are objective grounds for valuing better designed and more meritorious studies in making a risk assessment. Unfortunately, these reasons are often buried in documents hundreds of pages long and written in a scientific idiom that is far more challenging to read than a press release from an environmental activist group.

And, of course, when those press releases gush with admonitions that BPA is associated with everything bar leprosy and global warming, it is tempting to elide a statistical association into something that sounds much more connected and threatening.

The FDA has come out against restrictions on BPA, and the European Union, which regulates chemicals on the basis of the precautionary principle, has continued to defend the safety of the chemical. In order to explain why the institutions charged with ensuring public safety continue to affirm the safety of BPA in the face of an apparent deluge of warnings from some scientists and many environmental groups, it is essential for the media, not only to grasp what different kinds of statistical and investigative methods can and cannot say, but to report with the same avidity on studies that do not find a risk. The public’s faith in science and regulation cannot survive on a diet of bad news alone.

See also:

Why Journalism is Failing the Public on the Risk from Plastics
In the media's rush to report a health scare over bisphenol a (BPA), crucial facts and scientific consensus have gone missing.

View the Technorati Link Cosmos for this entry