10 Predictive Validity Examples

Predictive validity is a type of validity that refers to how well a person’s score on one variable predicts their score on a second variable.

The two variables we look at with predictive validity are:

• Predictor Variable: The predictor variable is sometimes called the explanatory variable, which predicts what will happen to the criterion variable
• Criterion Variable: The second variable is sometimes called the criterion variable, or response variable, which is what we’re trying to predict.

In the simplest terms, a correlation is calculated that assesses the degree of the relationship between the predictor and criterion variables.

If the correlation is very high (close to 1), then the relation between the two variables is quite strong, meaning there is high predictive validity.

If the test is 100% accurate at predicting scores on the criterion, then the correlation will be a perfect 1. That never happens.

Why use Predictive Validity?

Being able to predict a person’s performance in the future has tremendous value in many professions and industries.

For example, in human resources, being able to predict which employees have leadership potential based on their scores on a personality test would be of considerable value.

Likewise, universities would like to be able to predict a student’s GPA based on their SAT scores.

Examples of Correlations with High Predictive Validity

• Productivity Quizzes and Worker Productivity A quiz that tests how productive someone will be. The quiz asks questions about their personality and tests their ability to follow instructions. A person’s score on the personality test is the predictor variable and productivity is the criterion variable.
• Physical Attributes and Basketballer Success: Scouts look at physical attributes like height, standing reach, weight, wingspan, and hand length as predictor variables for what makes a successful defensive basketball player.
• Building Permits and the Housing Market: Here is a list of predictor variables that are related to the prices of houses (the criterion variable): number of new homes purchased, building permits awarded, interest rates on mortgages, and employment rate.
• Extraversion and Good Flight Attendants: Airlines try to employ flight attendants with two characteristics – extraversion and concern for others – as these are predictor variables that correlate with successful flight attendants.
• The In-Basket Job Simulation Test and Ability to Prioritize: This test gets a job applicant to sort items in an in-basket and prioritize the most pressing tasks. an applicant’s ability to prioritize (criterion variable) based on their performance in the in-basket exercise (predictor variable).
• Personality Predictors and a Long Lifespan: A study by Terracciano et al. (2008) found that being conscientious, emotionally stable, and active are predictor variables for living longer (the criterion valuable).
• The NFL Combine Test and Running Back Success: The NFL Combine is a yearly test of football players’ speed and strength. One element of this test that has statistical validity is sprint speed and the predicted performance of running backs.
• Parental Linguistic Ability and Child’s Language Skills: The language skills of a child are affected by how often parents talk to them. Here, the predictor variable is parents’ frequency of talk and the criterion variable is the language skills of a child.
• Air Pressure and Weather Forecasting: Modern weather forecasters use hundreds of predictor variables such as air pressure and time of year to improve the predictive validity of weather forecasts.
• Anger and Cardiovascular Disease: Anger, distrust, and antagonism have been found to be valid predictor variables for predicting likelihood of cardiovascular disease.

Detailed Explanations of the Above Examples

1. Productivity Quizzes and Worker Productivity

A company assigns a quiz that tests how productive someone will be. The quiz that asks questions about their personality and tests their ability to follow instructions.

Predictor Variable: Score on the productivity test.

Criterion Variable: Productivity in the workplace.

When a large corporation opens a new manufacturing plant, there may be thousands of people that apply. It would be very time-consuming and incredibly inefficient if the HR department had to conduct a 45-minute interview with each applicant.

Instead, they ask applicants to respond to an online survey of 50 questions that assesses one dimension of personality: ability to follow detailed instructions.

The HR department knows from years of testing at their other facilities that people that score high on this personality characteristic are also very productive assembly-line workers.

In this example, a person’s score on the personality test is the predictor variable and productivity is the criterion variable.

Of course, there are other factors that affect productivity, so the relationship between the predictor and criterion is not perfect. However, it is strong enough that the HR ends up making the right hiring decision a majority of the time.

2. Height and NBA Performance

Scouts look at physical attributes like height, standing reach, weight, wingspan, and hand length as predictor variables for what makes a successful defensive basketball player.

Predictor Variables: Height, standing reach, weight, wingspan, hand length.

Criterion Variable: Defensive performance in basketball.

Predicting which college hoops player will go on to have a successful NBA career has been the subject of study for decades. Scouts and head coaches, as well as TV analysts, all have their theories. Those theories are based on experience and being around the sport for many years.

Another approach is empirical. For example, Teramoto et al. (2018) measured athletes on a wide range of physical attributes such as wingspan, agility, vertical jump, and body fat percentage. The researchers then compared the data with playing good defensive in the NBA.

The results indicated that:

“…anthropometrics, including height without shoes, standing reach, weight, wingspan, and hand length…had positive, medium-to-large-sized correlations (r = 0.313-0.545) with defensive performance…” (p. 396).

Remember, the closer the correlation is to 1, the stronger the association. So, correlations of 0.313-0.545 are considered meaningful.

3. Building Permits and the Housing Market

Here is a list of variables that are related to the prices of houses: number of new homes purchased, building permits awarded, interest rates on mortgages, and employment rate.

Predictor Variables: Number of new homes purchased, building permits awarded, interest rates on mortgages, employment rate.

Criterion Variable: Cost of houses in a suburb.

The question is, which one those predictor variables is directly related to the overall well-being of the housing market (i.e., criterion variable)?

The answer is, all of the above. Each of the variables listed is a good predictor of the overall health of the housing market. There are a few others as well.

For an economist, being able to predict the economic performance of the country is essential. Therefore, they will accumulate data on hundreds of predictor variables, input them into a sophisticated computer program, and then examine the output.

They can even alter their economic forecasts by adjusting each of the predictor variables, or adding and subtracting some variables from the equation. Although it’s not an exact science, forecasts can be very accurate.

4. Personality Characteristics and Flight Attendants

Airlines try to employ flight attendants with two characteristics – extraversion and concern for others – as these are predictor variables that correlate with successful flight attendants.

Predictor Variables: Extraversion and concern for others.

Criterion Variable: Success as a flight attendant.

Being a flight attendant requires a very specific set of personality characteristics. First, being sociable and outgoing is important when working with passengers that may be cranky during long hauls. Secondly, in case of an emergency, a flight attendant must remain calm and exude a sense of being in control.

Fortunately for airlines, there are several personality scales available that can assess those exact characteristics.

Research indicates that flight attendants typically possess high levels of concern for others and are extraverts.

In this example, variables such as concern for others and extraversion should be good predictors of being a good flight attendant.

5. The In-Basket Job Simulation and Ability to Prioritize

The in-basket job simulation tests gets a job applicant to sort items in an in-basket and prioritize the most pressing tasks. an applicant’s ability to prioritize (criterion variable) based on their performance in the in-basket exercise (predictor variable).

Predictor Variable: Performance in the in-basket exercise.

Criterion Variable: Applicant’s ability to prioritize in the workplace.

This is an assessment tool used by employers to evaluate an applicant’s ability to prioritize. 

The procedure is fairly simple. The applicant is instructed to assume that they are a manager being faced with several deadlines for various tasks.

They are seated at a desk that looks very official and then asked to examine the various documents in the in-basket. Those documents include copies of emails, memos, meeting minutes, and various messages. The applicant is given a short period of time to read all of the documents and then place them in order of urgency.

The hiring manager will then score the applicant’s performance (i.e., predictor variable score).

Based on experience and data that has accumulated over years, the hiring manager will be able to estimate the applicant’s ability to prioritize (criterion variable) based on their performance in the in-basket exercise (predictor variable).

There are many variations and degrees of complexity for the in-basket simulation, but this basic version provides a good example.

6. Personality Predictors and Life Span

A study by Terracciano et al. (2008) found that being conscientious, emotionally stable, and active are predictor variables for living longer (the criterion variable).

Predictor Variable: Being conscientious, emotionally stable, and active.

Criterion Variable: Living longer.

There are a multitude of factors that affect how long a person lives. Biological variables include genetic predispositions to certain diseases. Behavioral variables include habits such as diet and exercise.

And then there are personality variables, such as anger/happiness and introversion/extraversion. These are all considered predictor variables of the criterion (longevity).

In a typical study, a large sample of people are assessed on a wide range of biological, behavioral, and psychological variables. Then, those individuals are tracked over the next 20-50 years. The researchers then compare the profile of the people that are still alive versus those that passed away.

For example, research by Terracciano et al. (2008) found that “In a large sample of generally healthy individuals followed for almost five decades, longevity was associated with being conscientious, emotionally stable, and active” (p. 621).

This study included a large number of predictor variables, but identified the three most important.

7. The NFL Combine and Running Back Performance

The NFL Combine is a yearly test of football players’ speed and strength. One element of this test that has statistical validity is sprint speed and the performance of running backs.

Predictor Variable: Sprint test scores.

Criterion Variable: Running backs’ football performance.

Professional football players in the US make a lot of money. Stars can pull in millions of dollars a year; some can actually earn tens of millions per year.

Unfortunately, for coaches and owners, being able to predict which college prospect is going to earn that money is like trying to predict the lottery.

So, each year, top college players attend the NFL combine and go through a range of physical assessments, such as how fast they can run 40 yards or how many times they can bench-press 220 lbs.

Why? Because head coaches are tied to tradition, even though these measures of athleticism have virtually no predictive validity of performance on gameday.

As stated by Kuzmits and Adams (2008), their study found “…no consistent statistical relationship between combine tests and professional football performance, with the notable exception of sprint tests for running backs” (p. 1721).

8. Parental Linguistic Input and Children’s Language Skills

The language skills of a child are affected by how often parents talk to them. Here, the predictor variable is parents’ frequency of talk and the criterion variable is the language skills of a child.

Predictor Variable: Parents’ frequency of talk

Criterion Variable: Child’s language skills

The more parents talk to their young children the better. Of course, the quality of those statements and emotional tone are also important.

However, decades of research in this area have consistently found very similar results: the language skills of a child are affected by how often parents talk to them.

How strong are the effects? Based on the results of a meta-analysis by Anderson et al. (2021), the effect may not be as strong as you would expect. Their examination of the research, which involved combining the effects of many studies into a single index, reveals that the relationship is moderate, but not strong.

This means that there are numerous other factors that affect children’s language skills.

In this example, the predictor variable is how often parents talk to their child, and the criterion variable is the child’s language skills. Although the predictive ability of parental linguistic input is moderate, it still plays an important role.

9. Air Pressure and Weather Forecasting

Modern weather forecasters use hundreds of predictor variables such as air pressure and time of year to improve the predictive validity of weather forecasts.

Predictor Variable: Air pressure and time of year

Criterion Variable: Tomorrow’s weather

Back in the early days of weather forecasting, the local weatherman simply called another weatherman in a city several hours away and asked, “how’s the weather?”

That “data” was then used to take a guess as to what kind of weather was coming soon. As you can probably imagine, accuracy was a bit disappointing.

However, today’s weather forecasting includes hundreds of predictor variables in the form of data collected from high-tech instruments all over the world, including satellites. Each predictor variable is used to estimate the criterion (i.e., weather).

Taking one variable at a time will produce a level of accuracy similar to that of 100 years ago. However, combing hundreds of predictor variables and inputting the data into a computer program can produce a level of accuracy that is substantially more reliable.

10. Anger and Cardiovascular Disease (CVD)

Anger, distrust, and antagonism have been found to be valid predictor variables for predicting the likelihood of cardiovascular disease.

Predictor Variable: Anger, distrust, and antagonism

Criterion Variable: Likelihood of cardiovascular disease

According to the U.S. Centers for Disease Control (CDC), heart disease (CVD) is the leading cause of death in the country.

One prominent symptom of CVD is atherosclerosis, which is when plaque accumulates in the walls of arteries. The accumulation can eventually prevent the passage of blood and cause a heart attack.

Doctors have long known that diets high in fat and a sedentary lifestyle are key predictors of CVD (criterion variable). But did you know that personality characteristics are also predictor variables?

“Although evidence is inconsistent, chronic feelings of anger, cynical distrust and antagonistic behavior are at least modestly associated with risk of both initiation and progression of CVD” (Suls, 2013, p. 538).

The key term there is “modestly associated.” That means two things. That there are other factors that affect CVD, and that anger is a serious issue that definitely plays a role.

Conclusion

Scientists want to understand the phenomena they study and be able to predict what will happen in the future. That means identifying the specific factors involved. In other words, what predictor variables are associated with the criterion?

This is often assessed by calculating a correlation between the predictor and criterion. The closer the correlation is to 1, the stronger the relation.

Given this research tool, we are now able to predict which applicants will be the most productive employees, which college basketball players will be the best at playing defense in the pros, how lifestyle and personality affect life-expectancy, and what the weather will be like over the next seven days.

Although predictive accuracy is never 100%, in some cases it can get remarkably close.

References

Anderson, N. J., Graham, S. A., Prime, H., Jenkins, J. M., & Madigan, S. (2021). Linking quality and quantity of parental linguistic input to child language skills: A meta-analysis. Child Development, 92(2), 484–501. https://doi.org/10.1111/cdev.13508

Cronbach, L. J. (1970). Essentials of Psychological Testing. New York: Harper & Row.

Kuzmits, F. E., & Adams, A. J. (2008). The NFL combine: does it predict performance in the National Football League? Journal of Strength and Conditioning Research, 22(6), 1721–1727. https://doi.org/10.1519/JSC.0b013e318185f09d

Suls, J. (2013). Anger and the heart: Perspectives on cardiac risk, mechanisms and interventions. Progress in Cardiovascular Diseases, 55(6), 538-47. https://doi.org/10.1016/j.pcad.2013.03.002

Teramoto, M., Cross, C. L., Rieger, R. H., Maak, T. G., & Willick, S. E. (2018). Predictive validity of national basketball association draft combine on future performance. The Journal of Strength & Conditioning Research, 32(2), 396-408.

Terracciano, A., Löckenhoff, C., Zonderman, A., Ferrucci, L., & Costa, P. (2008). Personality predictors of longevity: Activity, emotional stability, and conscientiousness. Psychosomatic Medicine, 70(6), 621-7. https://doi.org/10.1097/PSY.0b013e31817b9371