External validity refers to the degree to which the conclusions of a study can be generalized to other situations and populations.
If the results of one very specific study can apply to other contexts and groups of people, then that study has external validity.
This is a core concern in psychological research because so many studies take place in a laboratory setting on a sample of participants that fall within a very narrowly defined demographic.
There are many threats to external validity that every study must try to rectify when researchers design the procedures of the study and identify the group of participants.
Although no study is perfect, it is possible to study a specific issue utilizing a variety of procedures and samples over a long period of time that can help establish external validity.
Examples of External Validity
1. Sample Representativeness
One of the biggest threats to the external validity of research has to do with how well the sample of participants in the study represent the broader population.
Since a large portion of psychological studies involving people are conducted at universities, the participants are in a narrow age range and SES group. So, this leads to the question: do the actions of college students that are 18-22 years old also say something about the actions of older adults 45-50 years old that never attended college?
The two samples differ in age, occupation, generational characteristics, and numerous other variables. The sample in the study is not representative of the broader population, and therefore the study lacks external validity.
2. Cross-Cultural Limitations
A lot of psychological research is conducted in Western cultures. This doesn’t necessarily translate to Eastern cultural contexts.
Western cultures tend to be individualistic and egalitarian. Eastern cultures, generally speaking, are more collectivist and hierarchical.
This makes it difficult to generalize the conclusions of a study in the West to cultures in the East.
For example, research on leadership styles in the West have revealed that many workers admire and respect leaders with a participative leadership style. This style of leadership makes them feel valued and respected.
However, in a culture that is more hierarchical, this type of leadership style may be seen by workers as a sign of weakness. This makes the external validity of studies on leadership fairly limited to the cultural context in which they were conducted.
3. Ecological Validity
Ecological validity refers to how well conditions in the study mirror those in the real world. Lack of translatability to natural environments may be one of the most potent threats to external validity.
For example, researchers may be interested in understanding how the personality profiles of a married couple affect how they handle disputes. So, they recruit married couples to come into their office and discuss a range of issues—some mundane and some slightly controversial.
The problem is that the office setting has very little semblance to the couples’ home environment. At home, there are other things happening while couples interact with each other: noisy children or crying babies, visiting relatives or friends, various tasks, and household chores that are waiting to be done. The environment can be chaotic and stressful.
When the conclusions are drawn at the end of the study, there is a serious issue of how well the results apply to the natural environment of a couple’s home.
4. The Hawthorne Effect
The Hawthorne effect refers to the fact that research participants often change their behavior because they know they are in a study.
The original research that identified this phenomenon took place in the 1930s at the Hawthorne Works electric company.
The research changed various conditions of the work environment, such as lighting and breaks, to determine the effects on productivity. The results were reported as demonstrating that any change in the environment seemed to increase productivity, simply because the workers new they were under study (see McCambridge et al, 2014 for an updated analysis).
When people know they are in a study, they may behave in socially desirable ways to create a favorable impression, or, they may become less expressive because they feel anxious about being observed.
This means that the conclusions of the study cannot be generalized to the broader population. The Hawthorne effect is an example of an issue that can threaten both the internal and external validity of a study.
5. Time of Day
Time of day has a huge impact on results. For example, one study found that the best time to apply for parole is right after lunch!
For a variety of reasons, data collection may take place at a specific time of day. This could be due to difficulties scheduling participants or the availability of lab space; very practical issues that all researchers must deal with.
However, when the data collection procedures only occur at a specific time of day, it creates a threat to external validity. After all, people use their memory and attention all hours of the day and night.
It is not unreasonable to suggest that cognitive processes may be different in the morning versus in the late evening.
Therefore, the time of day that participants are studied can make it difficult to generalize the conclusions of the study to other times of the day.
6. Field Study Location
The results of a study in one location may not be translatable to a different location.
Field research takes place outside of a laboratory and somewhere in the natural environment. For example, research on consumer behavior may take place in a store or shopping mall. A study might include having trained observers record specific actions of consumers as they browse through a store and examine different products.
The results might support the study’s hypothesis regarding how consumers make purchase decisions.
However, because the study took place in a shopping mall, the conclusions may not be applicable to other settings.
For instance, since a large portion of consumers today shop online, can the results of a field study in a shopping mall be generalized to consumer behavior online?
In this example, the location of the field study actually limits the applicability of the conclusions to other situations.
7. Laboratory Research
A lot of psychological studies take place in a laboratory setting. Unfortunately, laboratory settings are more controlled than real life, so their external validity is often low.
Laboratories are used because researchers want to focus on the very specific relationship between one independent variable (IV) and one dependent variable (DV).
In order to do that, they must attempt to control as many extraneous variables in their study as possible. Unfortunately, the more extraneous variables controlled, the lower the external validity.
The real world is full of variables that are uncontrollable and unpredictable. At any given time, any of those factors could affect a person’s behavior. This is the nature of everyday life. For the scientist, however, this makes conducting research extremely difficult and can jeopardize the validity of the research.
All laboratory studies that control for extraneous variables have issues with external validity.
8. Temporal Validity
Do the results of a study conducted in the 1950s still apply today? That is the fundamental issue of temporal validity: the extent to which the findings of a study apply over time.
Not only do people change over time, but society does as well. Issues that were controversial in the 1950s may be considered mundane today. For example, research on conformity in the 1950s took place during a time in American society when there was a great deal of social pressure to act in accordance with strict gender roles.
Today, however, there are far fewer divisions of prescribed behavior between males and females. Therefore, research on gender roles in the 1950s may have very low temporal validity to today.
9. Psychological Realism
Because research in psychology is often conducted in controlled settings, it is often criticized for lacking realism. The setting and procedures are very artificial, and it can be hard to get participants fully engaged.
While the issues that psychologists study may be controversial or have a strong emotional impact, it is hard to recreate those conditions in the lab. This is referred to as psychological realism.
To address this issue, researchers try to create situations in the lab that have the same degree of emotional impact and psychological involvement as in real-world conditions.
The less psychological realism in a study, the lower the external validity.
10. Conceptual Replication
One of the biggest concerns in psychological research has to do with the lack of replication. If a study can’t be replicated, then its external validity is questionable.
Ideally, the results of a study produced by one research team can be conceptually reproduced by other research teams. Unfortunately, this rarely happens.
When attempting conceptual replication, a second research team may use slightly different procedures, independent and dependent variables, and sample of participants than the original researchers. The two research teams may even be located in different countries. Although there are many differences between the two studies, everything is conceptually similar.
When this is accomplished, there is greater confidence in the generalizability of the findings. Therefore, the conclusions have greater external validity.
Read Next: Internal Validity Examples
Conclusion
Psychological researchers would prefer if the conclusions derived from their studies can explain human behavior. That means the results of an individual study applies to the general population.
That’s not always the case. Sometimes research involves a small sample of participants that are much younger than most of the population, or differ in regard to other demographics. In other cases, the findings of a study may only apply to certain times of the day, specific situations and settings, or pertain to a certain culture.
Even if a study has great generalizability to an entire society, those findings may become less applicable 40 or 50 years later.
External validity is a significant challenge in psychological research, and although no single study can overcome all obstacles, over a period of time, using a variety of methods, confidence in the conclusions can become stronger.
References
Aronson, E., Wilson, T.D., & Brewer, m. (1998). Experimental methods. In D. Gilbert, S. Fiske, & G. Lindzey (Eds.), The Handbook of Social Psychology. (4th ed., Vol. 1, pp. 99–142.) New York: Random House.
Campbell, D. T., & Stanley, J. C. (1966). Experimental and Quasi-experimental Designs for Research. Chicago, IL: Rand McNally & Company
Cook, T. D. and Campbell, D. T. (1979). Quasi-Experimentation: Design and Analysis Issues for Field Settings. Houghton Mifflin, Boston.
Findley, M. G., Kikuta, K., & Denly, M. (2021). External validity. Annual Review of Political Science, 24, 365-393.
Flannelly, K. J., Flannelly, L. T, & Jankowski, K. (2018). Threats to the internal validity of experimental and quasi-experimental research in healthcare. Journal of Health Care Chaplaincy, 24, 1-24. https://doi.org/10.1080/08854726.2017.1421019
Kenny, D. A. (2019). Enhancing validity in psychological research. The American Psychologist, 74(9), 1018–1028. https://doi.org/10.1037/amp0000531
McCambridge, J., Witton, J., & Elbourne, D. (2014). Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. Journal of Clinical Epidemiology, 67(3), 267-277. https://doi.org/10.1016/j.jclinepi.2013.08.015
Dr. Cornell has worked in education for more than 20 years. His work has involved designing teacher certification for Trinity College in London and in-service training for state governments in the United States. He has trained kindergarten teachers in 8 countries and helped businessmen and women open baby centers and kindergartens in 3 countries.
This article was peer-reviewed and edited by Chris Drew (PhD). The review process on Helpful Professor involves having a PhD level expert fact check, edit, and contribute to articles. Reviewers ensure all content reflects expert academic consensus and is backed up with reference to academic studies. Dr. Drew has published over 20 academic articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education and holds a PhD in Education from ACU.