STATS at George Mason University
horiz bar
nav bar

What is the difference between controlled and observational studies?

The three classic types of studies in biomedical research are controlled (also called experimental), observational (also called epidemiological), and case-control. There are advantages and disadvantages to each type, and an awareness of these differences makes for a savvier consumer of public health information.

Suppose we want to see if a drug helps with headaches or not.

A controlled study on the drug would take two groups of people (let’s call them A and B) and give set A the drug and give set B a placebo. None of the participants would know which group they were in, so no one would know if he or she got the “real” drug. The researchers would then record whether set A really has reduced headaches as opposed to group B. The study is called controlled because each element of it is controlled by the people designing the study.

Typically, these are “double blind” experiments, which means that even the doctors or nurses interacting with the patients don’t know who receives the drug, and who receives the placebo. This precautionary method prevents the possibility that the patient picks up on subtle signs from the doctors, which could in turn influence his or her psychology, possibly having an effect on the outcome. It also prevents the possibility that the doctor or nurses give the patients different treatment based on their knowledge of whether they are receiving the tested drug or not.

An observational study would search out data about people already using the drug, and compare it to data on comparable people who are not using the drug. It is called observational because the tactic is to observe the effect of the drug (or an activity or a lifestyle, or whatever else) in the general population, without recruiting participants into a controlled environment.

A third kind of study (which comes up far less frequently in the media) is a case-control study. Case-control studies are most usually used to find side effects rather than to evaluate the effectiveness of a drug. They test the relationship between two occurrences to see if they are correlated by comparing/contrasting people with a certain trait, against similar people who do not exhibit the same trait.

For example, suppose we want to evaluate the relationship between miscarriage and coffee consumption in the first trimester of pregnancy. In this scenario, two groups are formed: one is made up of women who have had miscarriages and the other is made up of women who haven’t. The women may be “matched,” by controlling for factors such as age, economic status, medical care, number of pregnancies, etc. Then the two groups are questioned about their coffee intake in the first trimester, and their answers are compared. Depending on the outcome, there may be a correlation with heavy coffee drinking and miscarriage. One should be wary, however, of “recall bias” in a study like this. Women who have had miscarriages may be more likely to remember having consumed coffee, perhaps because they feel guilty or generally remember the first trimester better because of the trauma of the miscarriage.

Which is better - controlled or observational?
Both have their advantages, but a controlled study is generally considered more reliable. There are several reasons why: First, the people who participate in the study are randomly assigned to groups A and B. This means that there is no inherent bias as to who is taking the medicine based on the fact that some people opt to take the drug and others don’t. Second, in a controlled study, confounding factors such as age, race, weight, sex, etc. can all be accounted for in the design of the study itself. The statisticians involved typically calculate how many participants are needed in order to get a statistically significant result.

The other reason to favor controlled studies is that they can tell us something about the cause of an observed correlation. If the group of medicine-takers has a lower rate of headaches than the non-medicine users, then it is reasonable to conclude that the medicine is the reason for the disappearing headaches. In a controlled study, the medicine-takers are similar in every measurable quality (race, age, smoking status, etc.) as the non-medicine takers.

If the study were observational and found that those who take the medicine have a lower rate of headaches, one could only conclude that the medicine and fewer headaches are correlated. There may not be a direct causal relationship. For example, it could be that those who bother to take medicine also seek to reduce stress at work, and reduced job-related stress is the reason that their headaches disappear.

However, observational studies have two important advantages: first, they are typically much larger than controlled studies. One can collect data about thousands or even millions of people (as the census bureau does) and make conclusions that are more resilient just because there are so many people participating in the study.

Second, controlled studies pose a moral dilemma, that of taking away the right of the participant to make his or her own decisions. This is why, for example, all the studies on the harm of smoking are based on observational studies. Once there was sufficient reason to suspect that smoking was harmful, it would have been immoral to take two groups (randomly assigned) and have one smoke and the other not smoke in order to evaluate the rates of smoking-related diseases. Similarly, most studies on pregnant women and how their lifestyles affect their babies after they are born are observational; it would be hard to convince a randomly assigned group of women to drink a glass of wine each meal in order to see if this causes fetal alcohol syndrome or not.

Any type of study has to control for confounding factors. A study on light drinking while pregnant, for example, should account for the fact that women who drink are also more likely to smoke, and smoking causes fetal injury. Even in a controlled environment, confounding factors can creep in and affect the results (usually, because the study designers failed to account for a confounding factor). The art of conducting well-designed studies has its challenges, regardless of the nature of the study.



faq home

STATS home

email this page to a friend

next faq topic