- What is Internal Validity?
- there is a relationship between the treatment and the effect (in other words, the researcher must establish conclusion validity);
- the observed effect occurred after the treatment was implemented
- there are no plausible alternative hypotheses
Before you can discuss threats to internal validity, you must understand what it is. As the name suggests, internal validity is the kind of validity that only pertains to the specific implementation of the treatment or program that is being evaluated. Internal validity allows a researcher to claim that it is, in fact, the treatment or program that caused a change in the group that was treated.
In order for a researcher to make a causal claim about a treatment or program, he or she must be able to show that:
Criterion 3 is usually the most difficult for a researcher to meet. Especially in social research, where it is often impossible to insulate groups from exposure to their social environment or regulate their reaction to the experience of being evaluated, events, conditions and responses unrelated to the treatment or program under study can have an impact on the group. This can cause a researcher to mistakenly attribute an observable difference between the pre- and posttests to the treatment under study when in fact the unrelated event or response caused the change. In other words, a researcher will claim he or she has established a causal relationship when none exists.
What is a Single Group?
Well, obviously a single group is one group. In research terms, this means that no control group (a group that doesn't receive the program or treatment, also called a comparison group) is being used to serve as a a standard against which a researcher can compare the results of the treated group. Adding a control group can enable a researcher to eliminate many threats to internal validity, but it can often introduce new threats, such as those related to selection biases and social threats. Adding a control group can be difficult, expensive or inefficient for a field researcher. For this reason, researchers have to learn to be aware of and guard against single group threats to internal validity.
What kind of threats are there?
There are six threats to single group internal validity:
Read up on these descriptions. Once you think you've gained an understanding of how the threats to internal validity work, you might want to try playing the game:
Protecting against Threats
to Internal Validity
How can a researcher protect his or her evaluation against these threats? Well, one way, as stated above is to add a control group. This can enable a researcher to show that a threat such as testing is not as plausible if it had no effect on the control group. A threat like regression could be controlled for because a researcher could measure the amount of regression in the control group and compare that to the amount of regression in the treatment group.
Another way to protect against these threats would be to insulate the group under study. A threat such as history could be controlled if the researcher could make sure that the group was not exposed to any outside events that could make an impact on the study. Of course, this is rarely a practical option in field research.
A third way to protect against these threats is to be aware of them and take measurements of factors that might show if these threats are, indeed, likely to have had an impact on the study. For example, a researcher who is evaluating an AIDS awareness program could monitor the media during the period in which the program is being implemented to see if there are any major stories about AIDS that might influence the treatment group. This could help rule out a history threat. A researcher examining a math education program could measure how students are improving in their science and vocabulary knowledge at the same time. If they appear to have improved in math at a much greater rate than the other subjects, this could help the researcher to make a case against a maturation threat.
Randomized assignment to a group is a way to control for certain threats, such as regression, and possibly maturation (if the randomization is across age and experience, for example), but it can't really protect against history, testing, instrumentation or mortality.
Careful analysis of the data can guard against threats such as regression and mortality. For example, if you look at the pretest scores of participants who drop out of a program and find they are evenly distributed among the population, you can make a better argument against a mortality threat.
Of course, a researcher can guard against an instrumentation threat by using experienced observers and consistent, established instruments. However, this may not always be feasible or appropriate. A testing threat can be avoided simply by not administering a pretest, but then it is more difficult to establish that a change has occurred in the group. However, unobtrusive pre- and posttests will make the threat of testing less plausible.
Researchers try to guard against a variety of threats to internal validity by carefully designing their reseach. See Bill Trochim's site on Research Design for an explanation of how to protect your evaluation from falling prey to these potential research pitfalls.
References
Cook, T.D. & Campbell, D.T. (1979). Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston: Houghton-Mifflin.
Huck, S.W. & Sandler, H.M. (1979). Rival Hypotheses: Alternative Interpretations of Data Based Conclusions. New York: Harper & Row, Publishers.
Judd, C.M & Kenny, D.A. (1981). Estimating the Effects of Social Interventions. Cambridge: Cambridge University Press.
Trochim, W. (1996) "Knowledge Base" in BillTrochim's Center for Social Research Methods. http://trochim.human.cornell.edu/kb/kbhome.htm
Name That Threat | Trochim Home | Research Methods Tutorials
History | Instrumentation | Maturation | Mortality | Regression | Testing
Copyright © 1997 Wendy Martin. All rights reserved.