External validity

External validity is the validity of generalized (causal) inferences in scientific studies, usually based on experiments as experimental validity. [ Mitchell, M. & Jolley, J. (2001). "Research Design Explained (4th Ed)" New York:Harcourt.]

Inferences about cause-effect relationships based on a specific scientific study are said to possess external validity if they may be generalized from the unique and idiosyncratic settings, procedures and participants to other populations and conditions [Brewer, M. (2000). Research Design and Issues of Validity. In Reis, H. & Judd, C. (eds) "Handbook of Research Methods in Social and Personality Psychology." Cambridge:Cambridge University Press.] [Shadish, W., Cook, T., & Campbell, D. (2002). "Experimental and Quasi-Experimental Designs for Generalized Causal Inference" Boston:Houghton Mifflin.] Causal inferences said to possess high degrees of external validity can reasonably be expected to apply (a) to the target population of the study (i.e. from which the sample was drawn) (also referred to as population validity), and (b) to the universe of other populations (e.g. across time and space).

The most common loss of external validity comes from the fact that experiments using human participants often employ small samples obtained from a single geographic location or with idiosyncratic features (e.g. volunteers). Because of this, one can not be sure that the conclusions drawn about cause-effect-relationships do actually apply to people in other geographic locations or without these features.

Threats to external validity

"A threat to external validity is an explanation of how you might be wrong in making a generalization." [Trochim, William M. The Research Methods Knowledge Base, 2nd Edition.] Generally, generalizability is limited when the cause (i.e. the independent variable) depends on other factors; therefore, all threats to external validity interact with the independent variable.

* Aptitude-Treatment-Interaction: The sample may have certain features that may interact with the independent variable, limiting generalizability. For example, inferences based on comparative psychotherapy studies often employ specific samples (e.g. volunteers, highly depressed, no comorbidity). If psychotherapy is found effective for these sample patients, will it also be effective for non-volunteers or the mildly depressed or patients with concurrent other disorders?

* Situation: All situational specifics (e.g. treatment conditions, time, location, lighting, noise, treatment administration, investigator, timing, scope and extent of measurement, etc. etc.) of a study potentially limit generalizability.

* Pre-Test Effects: If cause-effect relationships can only be found when pre-tests are carried out, then this also limits the generality of the findings.

* Post-Test Effects: If cause-effect relationships can only be found when post-tests are carried out, then this also limits the generality of the findings.

* Reactivity (Placebo, Novelty, and Hawthorne Effects): If cause-effect relationships are found they might not be generalizable to other settings or situations if the effects found only occurred as an effect of studying the situation.

* Rosenthal Effects: Inferences about cause-consequence relationships may not be generalizable to other investigators or researchers.

External, internal, and ecological validity

In many studies and research designs, there may be a "trade-off" between internal validity and external validity: When measures are taken or procedures implemented aiming at increasing the chance for higher degrees of internal validity, these measures may also limit the generalizability of the findings.This situation has led many researchers call for "ecologically valid" experiments. By that they mean that experimental procedures should resemble "real-world" conditions. They criticize the lack of ecological validity in many laboratory-based studies with a focus on artificially controlled and constricted environments.External validity and ecological validity are closely related in the sense that causal inferences based on ecologically valid research designs often allow for higher degrees of generalizability than those obtained in an artificially produced lab environment. However, this is not always the case: Some findings produced in ecologically valid research settings may hardly be generalizable, and some findings produced in highly controlled settings may claim near-universal external validity. Thus, External and Ecological Validity are independent - a study may possess external validity but not ecological validity, and vice-versa.

Qualitative research

Within the qualitative research paradigm, external validity is replaced by the concept of transferability. Transferability is the ability of research results to transfer to situations with similar parameters, populations and characteristics. [Lincoln, Y.S. & Guba, E.G. (1986). But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. In D.D. Williams (Ed.), "Naturalistic evaluation" (pp. 73-84). New Directions for Program Evaluation, 30. San Francisco, CA: Jossey-Bass.]

ee also

*Statistical conclusion validity
*Internal validity
*Ecological validity
*Construct validity
*Content validity

Notes