STATS ARTICLES 2005
Divorce Study Authors Respond to Critique
November 17, 2005
Body of evidence supports our conclusions, say authors; groups not comparable in study, says Dr. Goldin
Divorce study authors respond to STATS How Bad is Divorce?
We totally support STATS' goal of assessing the accuracy of information reported by the press, but the STATS critique of the press coverage of our study of the effects of divorce on children (reported in the new book, Between Two Worlds: The Inner Lives of Children of Divorce) illustrates the difficulty of the task STATS has taken on.
The main problem is that no short press account of a study can report the full corpus of evidence on which conclusions are based, nor can any somewhat longer "opinion piece" do so. The lack of full reporting of evidence grows not only out of space limitations but out of the inappropriateness of trying to explain to lay readers such things as the results of regression analyses and the incorporation of unmeasured variables into causal models.
For instance, our study included multivariate analyses in which variables likely to affect both whether or not parents divorced and the outcome variables we studied were statistically held constant, and it included in-depth interviews as well as the survey data mentioned in the press reports. As the critic, Rebecca Goldin, correctly points out, only randomized experimentation can come close to providing absolutely conclusive evidence of cause and effect, and such experimentation is impossible in social research.
Does that mean that we should give up on trying to figure out what causes what? We think not. The kind of research that is possible can provide convincing, though not absolutely conclusive, evidence of causation. Tentative conclusions based on such evidence are better than the only alternatives, namely, guesses and conclusions based on preconceptions and prejudices.
Goldin takes Marquardt to task for making conclusions, in an article that Goldin correctly labels "an opinion piece," that go beyond the evidence yielded by our study. In particular, she doesn't like conclusions about effects of divorce on adults, since our study didn't deal with adults. In fact, there is a huge amount of evidence on the effects of divorce on adults, so much evidence that the fact that divorce is typically painful to one or both ex-spouses is common knowledge. Apparently, Goldin would remove references to common knowledge, as well as all "opinions," from opinion pieces. Good luck in getting editors and journalists to accept the standard implied by that critique.
If opinion pieces had to be based entirely on absolutely conclusive evidence, and only on the evidence yielded by one study, they would disappear from the journalistic literature, and that, we believe, would be a major loss.
Norval D. Glenn
Stiles Professor in American Studies
Department of Sociology
University of Texas at Austin
Institute for American Values
New York City
Dr. Goldin responds
As scientists, we agree that it would be a loss to society if research on social phenomena were shelved because we can’t do randomized experimentation. However, we disagree on what conclusions can be drawn from this study on divorce.
What the study did do in an exemplary fashion is look into the long-lasting effects of divorce on children. Even the author’s statement in the response – that it is a widely recognized fact that adults also are pained by divorce – is certainly a conclusion that can be reached by surveys and interviews with divorced people (though this particular study did not include parents).
I do not challenge these results; indeed, I think them extremely important findings. Only when we recognize the pain that divorce wreaks can we understand, and hopefully better respond to these problems – and a possible response could be “preventative”. I agree with the authors that the problem may not lie only with the reasons behind the divorce, but the divorce itself may be a factor.
Where I disagree is whether we can justly compare those people whose parents had a “good” divorce – one in which there was overall civility between the divorcés – and those whose parents had “low-conflict” yet “unhappy” marriages, but chose not to divorce. My argument is that among people who have low-conflict, unhappy marriages, the ones who decide to divorce are not comparable to the ones who decide to stay married.
The Glenn-Marquardt argument is that they controlled for enough factors using statistical methods to be sure that the two groups are comparable. The problem is that, practically speaking, it is impossible to control for all factors.
I would argue that, even if all “measurements” of unhappiness and conflict were comparable, the very fact that one group decided to divorce and the other didn’t reflects a difference in mentality that, perhaps, was simply not measured by surveys and interviews. Factors such as “mentality” and “values” are not easily assessed, especially if you are only surveying the kids.
In order to conclude that divorce is an independent factor in the emotional trauma experienced by the young adults interviewed for the study, one needs to control for an assortment of factors that are, in and of themselves, associated with divorce.
This brings up an important statistical question regarding the “causal pathway” and “independent variables.” If, for example, being religious is associated with lower divorce rates, then should religiosity be a variable that is controlled for? If you do not control for religiosity, you might measure an effect of divorce that is actually a reflection of the religious level of the family.
On the other hand, if you do control for religiosity (which is to say, you adjust your statistics so that people who divorce and people who stay married are equally religious), then you might not pick up an effect due to religiosity that is in turn due to divorce. There is no clear “right” answer to whether to control for this variable. And there may be many variables that are like it.
This difficulty in making causal connections is widespread. I'll illustrate this with another example. If diabetes is associated with obesity, then should we control for diabetes when we try to assess the mortality rate due to obesity? It depends on the assumptions you make about the “causal pathway” – if diabetes contributes to obesity, then diabetes should not be controlled for. If diabetes is an independent occurrence – perhaps due to some of the same root causes as the obesity – then it should be adjusted for.
I agree that opinion pieces should have room for comments such as Marquardt’s. However, in a piece written about scientific results, we welcome more caution in suggesting to the public that the science unfalteringly supports the opinion. My opinion is that it does not.