Regression to the Mean
Regression to the mean is my favorite threat to internal validity. Why? Well, because it shows that, in the end, people are really not so different, and researchers can just throw any old test at a group and use that to make claims about them. Regression to the mean gives hope to those at the bottom of any scale, and forces those at the top to realize they are not as superior as they might have thought. It forces researchers to think twice before they make causal claims about the treatments they study.
So, now are you intrigued enough to continue? Good, because it can be a little difficult to explain. The first thing you need to realize when trying to understand regression to the mean is that testing measures are rarely perfectly reliable. Because of this, pretest scores on a measure will not correlate perfectly with posttest scores. The individual scores on the same two tests taken on differnt occasions will almost always vary.
Measurement error plays a big role in regression. An observed score is comprised of the test-taker's true score plus the degree of measurement error. For example, on a test of reading comprehension, the true score is that person's true reading comprehension ability. The error can consist of conditions that have a negative impact--sickness, exhaustion, discomfort--or a positive impact--lucky guessing, ttest-taker recently read an article similar to the example and therefore can answer some of the questions even without reading carefully. The next time he or she takes the test, that error condition is likely to be different. So, while one's true ability may not change, the score on the test will be different.
So, how does this relate to my egalitarian description of this threat? Well, the amout of difference that is likely to be observed the second time a person is tested is related to how far above or below the group mean that person's score was, and the degree of pretest-posttest reliability of the measure. In other words, a person who scored poorly on a pretest is likely to score better on a posttest, while a person who scored well is likely to have a lower score on the posttest. Both will have scores that are closer to, or regress toward, the group mean. The highest and lowest scorers will regress toward the mean at a higher rate than those who scored close to the mean. There will be a higher degree of regression for unreliable measures than for more reliable ones.
So, how is this a threat to internal validity? Suppose you gave a pretest to a group, say a measure of self-esteem, and, based on their scores, you took those who appeared to have the lowest self-esteem and put them into what you believed was a self-esteem enhancing program. When you measure their self-esteem at the end of the program, you notice that their scores are higher than they were on the pretest. You decide it was your program that caused this effect. In fact, their higher scores could simply be a case of regression to the mean. In other words, because they were at the very lowest end of the scale, they would have shown an improvement even if you had not given them any treatment.
How do you think a comparison group could alleviate this threat to internal validity?
Back
Home | Name That Threat | Trochim Home | Research Methods Tutorials
History | Instrumentation | Maturation | Mortality | Testing
Copyright © 1997 Wendy Martin. All rights reserved.