Experimental design refers to a research methodology that allows researchers to test a hypothesis regarding the effects of an independent variable on a dependent variable.
There are three types of experimental design: pre-experimental design, quasi-experimental design, and true experimental design.
Experimental Design in a Nutshell
A typical and simple experiment will look like the following:
- The experiment consists of two groups: treatment and control.
- Participants are randomly assigned to be in one of the groups (‘conditions’).
- The treatment group participants are administered the independent variable (e.g. given a medication).
- The control group is not given the treatment.
- The researchers then measure a dependent variable (e.g improvement in health between the groups).
If the independent variable affects the dependent variable, then there should be noticeable differences on the dependent variable between the treatment and control conditions.
The experiment is a type of research methodology that involves the manipulation of at least one independent variable and the measurement of at least one dependent variable.
If the independent variable affects the dependent variable, then the researchers can use the term “causality.”
Types of Experimental Design
1. Pre-Experimental Design
A researcher may use pre-experimental design if they want to test the effects of the independent variable on a single participant or a small group of participants.
The purpose is exploratory in nature, to see if the independent variable has any effect at all.
The pre-experiment is the simplest form of an experiment that does not contain a control condition.
However, because there is no control condition for comparison, the researcher cannot conclude that the independent variable causes change in the dependent variable.
- Action Research in the Classroom: Action research in education involves a teacher conducting small-scale research in their classroom designed to address problems they and their students currently face.
- Case Study Research: Case studies are small-scale, often in-depth, studies that are notusually generalizable.
- A Pilot Study: Pilot studies are small-scale studies that take place before the main experiment to test the feasibility of the project.
- Ethnography: An ethnographic research study will involve thick research of a small cohort to generate descriptive rather than predictive results.
2. Quasi-Experimental Design
The quasi-experiment is a methodology to test the effects of an independent variable on a dependent variable. However, the participants are not randomly assigned to treatment or control conditions. Instead, the participants already exist in representative sample groups or categories, such as male/female or high/low SES class.
Because the participants cannot be randomly assigned to male/female or high/low SES, there are limitations on the use of the term “causality.”
Researchers must refrain from inferring that the independent variable caused changes in the dependent variable because the participants existed in already formed categories before the study began.
- Homogenous Representative Sampling: When the research participant group is homogenous (i.e. not diverse) then the generalizability of the study is diminished.
- Non-Probability Sampling: When researchers select participants through subjective means such as non-probability sampling, they are engaging in quasi-experimental design and cannot assign causality.
3. True Experimental Design
A true experiment involves a design in which participants are randomly assigned to conditions, there exists at least two conditions (treatment and control) and the researcher manipulates the level of the independent variable (independent variable).
When these three criteria are met, then the observed changes in the dependent variable (dependent variable) are most likely caused by the different levels of the independent variable.
Of course, no study is perfect, so researchers must also take into account any threats to internal validity that may exist such as confounding variables or experimenter bias.
- Heterogenous Sample Groups: True experiments often contain heterogenous groups that represent a wide population.
- Clinical Trials: Clinical trials such as those required for approval of new medications are required to be true experiments that can assign causality.
Experimental Design vs Observational Design
Experimental design is often contrasted to observational design. Defined succinctly, an experimental design is a method in which the researcher manipulates one or more variables to determine their effects on another variable, while observational design involves the observation and analysis of a subject without influencing their behavior or conditions.
Observational design primarily involves data collection without direct involvement from the researcher. Here, the variables aren’t manipulated as they would be in an experimental design.
An example of an observational study might be research examining the correlation between exercise frequency and academic performance using data from students’ gym and classroom records.
The key difference between these two designs is the degree of control exerted in the experiment. In experimental studies, the investigator controls conditions and their manipulation, while observational studies only allow the observation of conditions as independently determined (Althubaiti, 2016).
Observational designs cannot infer causality as well as experimental designs; but they are highly effective at generating descriptive statistics.
|Observational Design||Experimental Design|
|Definition||A research approach where the investigator observes without intervening, often in natural settings.||A research approach where the investigator manipulates one variable and observes the effect on another variable.|
|Control||The researcher does not control or manipulate variables, but only observes them as they naturally occur.||The researcher has complete control over the variables being studied, including the manipulation of the independent variable.|
|Intervention||There is no intervention or manipulation by the researcher.||The researcher intentionally introduces an intervention or treatment.|
|Purpose||To identify patterns and relationships in naturally occurring data.||To determine cause-and-effect relationships between variables.|
|Examples||Observing behaviors in their natural environments, conducting surveys, etc.||Conducting a clinical trial to determine the efficacy of a new drug, using a control and treatment group, etc.|
|Strengths||Useful when manipulation is unethical or impractical; Can provide rich, real-world data.||Can establish causality; Can be controlled for confounding factors.|
|Weaknesses||Cannot establish causality; Potential for confounding variables.||May lack ecological validity (real-world application); Can be costly and time-consuming.|
|Data Collection||Typically collected in the form of qualitative data, but can also be quantitative.||Typically collected in the form of quantitative data, but can also be qualitative.|
For more, read: Observational vs Experimental Studies
Generally speaking, there are three broad categories of experiments. Each one serves a specific purpose and has associated limitations. The pre-experiment is an exploratory study to gather preliminary data on the effectiveness of a treatment and determine if a larger study is warranted.
The quasi-experiment is used when studying preexisting groups, such as people living in various cities or falling into various demographic categories. Although very informative, the results are limited by the presence of possible extraneous variables that cannot be controlled.
The true experiment is the most scientifically rigorous type of study. The researcher can manipulate the level of the independent variable and observe changes, if any, on the dependent variable. The key to the experiment is randomly assigning participants to conditions. Random assignment eliminates a lot of confounds and extraneous variables, and allows the researchers to use the term “causality.”
For More, See: Examples of Random Assignment
Baumrind, D. (1991). Parenting styles and adolescent development. In R. M. Lerner, A. C. Peterson, & J. Brooks-Gunn (Eds.), Encyclopedia of Adolescence (pp. 746–758). New York: Garland Publishing, Inc.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings. Boston, MA: Houghton Mifflin.
Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13(5), 585–589.
Matthew L. Maciejewski (2020) Quasi-experimental design. Biostatistics & Epidemiology, 4(1), 38-47. https://doi.org/10.1080/24709360.2018.1477468
Thyer, Bruce. (2012). Quasi-Experimental Research Designs. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195387384.001.0001