CHEAT SHEET: A QUICK REFERENCE GUIDE TO THE THREATS TO CONSTRUCT VALIDITY

• INADEQUATE PREOPERATIONAL EXPLICATION OF CONSTRUCTS:
  In English: You didn't do such a hot job of operationally defining your constructs.
  So What? Your critics could accuse you of not clearly thinking through your constructs.
  Our Example: Susie needs to think through her constructs of parent involvement and parenting classes; among other things she needs to consult with experts so she is looking at what she thinks she is looking at.

• MONO-OPERATION BIAS:
  In English: You only applied one version of your program.
  So What? One could argue that the results of your study only reflect the particular version of the program that you used.
  Our example: Susie needs to think about employing more than version of her parenting program to strengthen her argument against this threat.

• MONO-METHOD BIAS:
  In English: You only used one measure to look at your construct of interest (example -one questionnaire, one observation, etc).
  So What? Maybe you didn't measure what you thought you measured...
  Our example: In this case, Susie needs to measure parent involvement in more than one way -- perhaps through a questionnaire and an observation.

• INTERACTION OF DIFFERENT TREATMENTS:
  In English: It was really a combination of things -- your program and other factors -- that produced the desired effect.
  So What? You think you did more than you really did (stop patting yourself on the back so much!).
  Our example: It may have been a combination of things, including parents' participation in other organizations, in conjunction with Susie's program that yield the desired effect.

• INTERACTION OF TESTING AND TREATMENT:
  In English: Your testing, in combination with the actual program, produced the desired effects.
  So What? You are labeling incorrectly -- when you say program you mean just the program, when in actuality the program includes the testing.
  Our Example: By pretesting her parents, Susie may raise their level of awareness of parent involvement. It may be because of this heightened awareness, and not necessarily Susie's program, that the desired effects emerge.

• RESTRICTED GENERALIZABILITY ACROSS CONSTRUCTS:
  In English: Things happened that you didn't necessarily predict and although your program had a positive effect on the construct of interest, that "possessiveness" may not be generalizable across constructs.
  So What? BE CAREFUL!!
  Our Example: Susie's parenting classes may increase the level of parent involvement, but they may also increase the amount of marital distress (as one spouse may accuse the other of not spending enough time with the kids). This threat reminds us to be careful about whether the desired "goodness" or "badness" of an outcome can be generalizable across constructs.

• CONFOUNDING CONSTRUCTS AND LEVELS OF CONSTRUCTS:
  In English: It wasn't necessarily that the program or treatment you applied did or didn't work, but rather that the AMOUNT or LEVEL of the treatment didn't work.
  So What? Maybe you can start patting yourself on the back again -- it may be better than you think!
  Our Example: If Susie's data analysis showed that her program had no effect, she may become unnecessarily discouraged. It may have simply been that she did not offer ENOUGH parenting classes, but the concept of parenting classes is still a good one.

• HYPOTHESIS GUESSING:
  In English: The members of your study guessed what you were trying to examine and based their behavior on this guess.
  So What? You think your program "did the trick", when in actuality just KNOWING what your program was SUPPOSED to do "did the trick".
  Our Example: If the parents guessed that Susie was looking at parent involvement, they may alter their level of involvement accordingly.

• EVALUATION APPREHENSION:
  In English: People have this burning desire to look cool, smart, knowledgeable, etc.
  So What? Because people are trying to look socially acceptable, they may not be acting in a way that reflects reality when they are not participating in a research project.
  Our Example: The parents in the study may say or act like they are highly involved with their kids, just to "look good", regardless of the effect of the program.

• EXPERIMENTER EXPECTANCIES:
  In English: Without knowing it, you encourage your subjects to respond in certain way.
  So What? Again, participants are not acting or responding as they would in reality, distorting your data.
  Our Example: Susie may act overly enthusiastic in regards to parent involvement and therefore influence the behavior of her parents.
  

  If you have had enough fun on this page for one day, and you want to continue with the engaging saga of Susie, click here Threats to Validity .