External validity refers to the extent to which the results of a study can be generalized or applied to settings, people, times, and measures other than the ones used in the study.
A study with high external validity will be generalizable beyond the specific conditions or participants of the original study. This means the study will likely be of great value in the real world because it means it is likely to be applicable to other people’s specific contexts, as well.
Definition of External Validity in Psychology
Generally, we think of external validity in psychology as a measure of how likely a cause-and-effect relationship will hold in settings external to the setting in which the study took place.
If you are writing a research paper, methodology chapter, or essay on external validity, be sure to use a scholarly definition.
Here are two that you could use:
- “External validity is the extent to which there is confidence that a study’s result may be generalized to hold over variations in populations, settings, treatments, and outcomes.” (Kviz, 2019)
- “External validity is defined as the extent to which results generalize to other participants, settings, follow-up times, and so on.” (Hagger-Johnson, 2014)
It is important to know how this differs from internal validity, outlined below.
Internal vs External Validity
If external validity refers to the extent to which we can be sure the cause-and-effect relationship between variables will hold up in external settings, internal validity refers to the extent to which we can sure it holds up within the study itself (Kenny, 2019; Kviz, 2019).
In other words, a poorly designed study may make claims to cause-and-effect that are inaccurate or misguided:
Internal validity is assessed as the extent to which plausible alternative explanations may be ruled out that a change in a dependent variable is not caused by the independent variable.” (Kviz, 2019)
These problems generally emerge through poor research design, such as failure to control variables, causing the third variable problem. This is a problem where a third variable, known as the confounding variable, is exerting an effect on the dependent and/or independent variable unknown to the researchers.
If a study has low internal validity then its results will be questioned, while a study with high internal validity is considered to have sound results.
Read More: Threats to Internal Validity
Types of External Validity
There are two types of external validity: population validity and ecological validity.
1. Population Validity
Population validity describes the degree to which the cohort examined in the study reflect the broader population (Kenny, 2019).
For example, if researchers want to test the viability of a new pill that will be rolled-out across the entire nation, then their research participants need to be a representative sample of the whole population (Findley, Kikuta & Denly, 2021). There would need to be people of all ages, genders, and races, ideally of equal proportion to the national population, included in the trial.
Meanwhile, if researchers are testing a new intervention that is exclusively intended for people who suffer from anxiety, then we would expect that the population in the research study would all have anxiety. We wouldn’t want to include people without anxiety in the study, because they would not reflect the target cohort in the real world.
2. Ecological Validity
Ecological validity describes the degree to which the setting in which the study takes place reflects the real-world environment (Andrade, 2018).
Oftentimes, lab-based studies suffer from low ecological validity because it is difficult to replicate real-world conditions in a lab setting.
Imagine, for example, a study where parents’ focus is observed in an observation room. In this room, there are no real-world distractions such as phones, television, or children to look after.
This study doesn’t reflect the real-world setting at all. The results would likely not truly reflect the actual degree of focus these parents have in a real-world environment. So, in this context, perhaps an observation study would make more sense.
There is often a trade-off between ecological validity and the ability to control variables. A controlled environment can improve the likelihood of identifying how well an independent variable can influence the dependent variable because it controls for extraneous variables; however, when controlling for extraneous variables in a manufactured setting, we sometimes don’t see what actually would take place in the real world (Andrade, 2018). So, multiple studies with different designs may need to be conducted to get a holistic understanding of the topic.
Threats to External Validity
Kviz (2019) points to four key threats to external validity. In each case, the threat emerges because elements of the study would be unrepresentative in other contexts.
- Unrepresentative units: The units, by which Kviz means people or groups within the study, may not represent the broader population. For example, if the participants in a study of school performance were all boys between ages 15 and 17, then there would be little justification for a claim that these results would be valid in mainstream co-educational K-12 classrooms. The ages and genders of the participants don’t match the setting to which you would be wanting to generalize the claims.
- Unrepresentative setting: Similarly, the setting of a study may not match the generalizable population. To take the above example, if the school that the study took place in was an all-boys’ school, then this is unrepresentative of most mainstream state schools which are co-educational. As such, external validity would be diminished. Similarly, a study in a lab setting is not likely representative of the complexity of a real-world setting.
- Unrepresentative treatment: A treatment is any manipulation of an independent variable in order to determine its effects on a dependent variable to identify a causal relationship. If the treatment in a test setting does not reflect practice settings, then external validity is diminished. To take the above school example, if our study involved a classroom intervention in which students got a lesson from a highly-trained specialist, then we can’t expect generalizability unless populations outside of the study also get an equivalent lesson from an equivalently trained specialist.
- Unrepresentative outcome: Generalizability may not be likely if the outcome is not measured in the same way in the test and practice settings. For example, researchers may use a different measurement tool in the lab setting than practitioners do in a real-life setting; or, practitioners may not be able to measure outcomes at the same time and with the same rigor in real-life settings.
Read About More Threats to External Validity Here
Strategies for Increasing External Validity in Psychological Studies
Enhancing external validity is crucial for ensuring that research findings can be generalized beyond the specific settings, groups, or conditions of a study.
Here are five strategies to increase external validity:
- Random Sampling: This involves selecting participants for a study from the larger population in a way that every individual has an equal chance of being chosen (Gall, Gall, & Borg, 2014). By ensuring that the study sample is representative of the larger population, the results can be more confidently generalized to the broader group.
- Using Real-world Settings (Field Experiments): Instead of conducting experiments in controlled laboratory settings, researchers can carry out their studies in natural, real-world settings where participants behave more naturally (Darlington & Scott, 2015). The findings are more likely to be applicable in everyday situations when the study context mirrors real-world conditions.
- Replication: This involves repeating the same study in different settings, with different populations, or under different conditions (Kenny, 2019). If similar results are obtained across various replications, it strengthens the belief that the findings are not limited to the specific conditions of a single study and can be generalized more broadly.
- Using Heterogeneous Participants: Instead of focusing on a narrow or specific group of participants, researchers can include a diverse range of participants in terms of age, gender, ethnicity, socioeconomic status, etc., that might best reflect the real world (Kenny, 2019). A diverse participant pool ensures that findings are not just applicable to a specific subgroup but can be generalized to a broader population.
- Cross-cultural Studies: Researchers conduct studies across different cultures or countries to see if the findings hold true in different cultural settings (Gall, Gall, & Borg, 2014). This ensures that the results are not bound by cultural nuances and can be generalized across different cultural contexts.
Conclusion
While there are strategies to enhance external validity in research (explained above), achieving perfect external validity is very challenging.
Often, psychology researchers have to balance between internal validity (ensuring that the study measures what it intends to measure without interference from other variables) and external validity.
The key is to always be aware of the potential limitations and address them as much as possible in the study design and interpretation of results.
References
Andrade, C. (2018). Internal, external, and ecological validity in research design, conduct, and evaluation. Indian journal of psychological medicine, 40(5), 498-499.
Darlington, Y., & Scott, D. (2015). Understanding qualitative research and ethnomethodology. London: Sage.
Findley, M. G., Kikuta, K., & Denly, M. (2021). External validity. Annual Review of Political Science, 24, 365-393. (Source)
Gall, M. D., Gall, J. P., & Borg, W. R. (2014). Applying educational research: How to read, do, and use research to solve problems of practice. Sydney: Pearson.
Hagger-Johnson, G. (2014). Introduction to Research Methods and Data Analysis in the Health Sciences. Taylor & Francis.
Kenny, D. A. (2019). Enhancing validity in psychological research. The American Psychologist, 74(9), 1018–1028. https://doi.org/10.1037/amp0000531
Kviz, F. J. (2019). Conducting Health Research: Principles, Process, and Methods. SAGE Publications.
Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]