15 Survivorship Bias Examples

Survivorship bias is when the entities in a sample exist because they have survived an elimination process. This is a type of sampling bias and can unduly influence judgments about a given phenomenon. Researchers may focus their attention on cases that have survived a criterion instead of considering all cases originally involved.

The famous statistician Abraham Wald is credited with discovering the survivorship bias when he was examining data on fighter plane survival rates.

He realized that the military was only considering planes that survived in their analyses, but more accurate data needed to include the planes that were shot down as well.

The survivorship bias can lead to overly optimistic conclusions. The entities that survived may have done so due to some unique, extra resilient quality that is not representative of typical examples.

Survivorship bias exists in an incredibly wide array of topics, from basic psychological and medical research, to economics, the financial performance of mutual funds, and survival characteristics of fighter planes.

Survivorship Bias Examples

• The life of professional baseball players looks glamorous, and profitable…until you find out how many professional players are stuck in the minors for years making very little money and never make it to the pros.
• We think that appliances were made better in the past because grandma still has her same stove and fridge. However, we aren’t considering the hundreds of other appliances she owned that failed.
• A team of biologists are studying trees that have survived infestation from pine beetles. However, they should also be studying the trees that did not survive.
• Assessing employee satisfaction of current employees may paint a rosy picture because those that were unhappy quit.
• Testimonials about a weight-loss product sound convincing. But, if we hear from all the people that tried the product and didn’t lose weight, we discover the diet only works for a small percentage of people and will do little to change the U.S. obesity rate.
• Being an entrepreneur looks like a great way to get rich. Unfortunately, we only see those that were successful in the news while the other 95% went bankrupt.
• A software company sends a satisfaction survey to its long-term corporate clients. However, there are lots of corporations that stopped using the software because they were unhappy with its performance.
• A person starts drinking a special kind of tea made from an exotic fungus because a recent study found it prevents macular degeneration. Unfortunately, the studies that showed it increases degeneration were never published.
• A researcher is studying how bees withstand droughts and continue pollinating crops. Unfortunately, the sample is not representative of the bee population because only bees that have survived are studied.
• People will prefer a medical treatment described as 90% effective over one that is described as having a 10% mortality rate (this is also an example of the framing bias).

Detailed Examples

1. Everyone is Beautiful

What we see on social media are the most popular videos. Those videos are often popular because the people in them are exceptionally attractive.

The videos of people that are not so attractive are not clicked on as much or shared with other people. That means that after a while, the videos that remain are the ones that have survived a process of elimination.

When people in society are constantly exposed to videos of abnormally attractive individuals, it can distort their perception of reality. We begin to believe that everyone is beautiful because that is what we see most of the time.

However, if we could also see the people in videos that did not “make the cut”, then we would have a much more accurate impression of what is a typical level of attractiveness.

2. Music was Better in the 60s

Songs from the 60s that we hear today are only the best and most popular tunes from that time. All the bad music that existed during that time is simply not played.

That means that our perception of music from the 60s is heavily influenced by the survivorship bias. Because we are only exposed to the best music from that era, it seems that it was all great.

In reality however, to make a more accurate assessment, we need to hear music from the entire population of music from the 60s; not just a subset of the best. Unfortunately, that would mean listening to a lot of tracks on albums that, well, maybe don’t sound so great.

3. All Asians are Good at Math

There is a common perception in North America that everybody from Asia is a math genius. That’s because the smartest students in math class seem to come from that region.

Although this is a flattering impression for Asian people, it might not be quite so accurate.

The Asian students we see in North America are those that have survived a grueling process of elimination. They have taken and excelled on numerous tests over a period of several years. Those that did not perform well on those tests did not make the final cut to attend universities abroad.

So, it seems like Asian students are super good at math because we never see those that are not. This is a straightforward case of survivorship bias (see also: the ecological fallacy).

4. Publication Bias

There is a big problem in research. As it turns out, a lot of studies that are conducted are never published. That means scientists are only reading research that has survived a publication process that eliminates a lot of studies.

Song et al. (2013) explain the problem concisely: “In general, studies with statistically significant or positive results are more likely to be published than those with nonsignificant or negative results” (p. 72).

This is a quite serious matter. Government officials sometimes make policy decisions based on the opinions of experts in a particular area, such as healthcare and economics. Hospitals and doctors often implement treatment based on what the research says is most effective.

However, what if there are studies that conclude a treatment is not effective, or that a certain policy is actually detrimental, but those studies are never published?

5. Performance of Mutual Funds

Survivorship bias also plays a role in the stock market. When speaking with current mutual fund managers, we may get the impression that performance is much better than is actually the case.

For example, research on mutual funds only use data regarding mutual funds that exist today. That research does not include data on funds that no longer exist.

Mutual funds that survived a recession or other economic hardships do not represent the entire population of funds. Although some funds survived, many others did not.

This skews the results of performance. Funds that were shut down are not included in the analysis. So, it appears as though mutual funds are a much better investment than they actually are if we consider the whole picture.

This fact has been supported in numerous studies. Brown et al. (1992) stated that “We have shown that truncation by survival has a measurable impact on the observed returns of those managers who survive the performance cut” (p. 576).

Conclusion

Survivorship bias is a tricky phenomenon to catch. The problem arises because we need to take into account data that we do not see.

For example, it may appear that mutual funds are a great investment. However, that’s because we are unaware of the number of funds that have failed and no longer exist.

The ramifications can be even more serious if the issue is medical care. Studies that fail to show that a treatment is effective are usually not published. If they are not published, then doctors can’t read them (obviously).

This can, and has, led to situations in which patients have been given a treatment that is not only ineffective, but even dangerous. 

But since you have now read this article on survivorship bias, you can apply this knowledge in your daily life and possibly stop yourself from making a bad decision. Therefore, the helpfulprofessor is good for your health.

References

Brown, S. J., Goetzmann, W., Ibbotson, R. G., & Ross, S. A. (1992). Survivorship bias in performance studies. The Review of Financial Studies, 5(4), 553-580.

Mangel, M., & Samaniego, F. J. (1984). Abraham Wald’s work on aircraft survivability. Journal of the American Statistical Association, 79(386), 259-267.  https://doi.org/10.1080/01621459.1984.10478038

Mlinarić, A., Horvat, M., & Šupak Smolčić, V. (2017). Dealing with the positive publication bias: Why you should really publish your negative results. Biochemia Medica, 27(3), 030201. https://doi.org/10.11613/BM.2017.030201

Song, F., Hooper, L., & YK, L. (2013). Publication bias: What is it? How do we measure it? How do we avoid it? Open Access Journal of Clinical Trials, 5, 51-81. https://doi.org/10.2147/OAJCT.S34419

Baumeister, R. F., Bratslavsky, E., Finkenauer, C., & Vohs, K. D., (2001). Bad is stronger than good. Review of General Psychology, 5(4), 323–370.

Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453-458.

Beratšová, A., Krchová, K., Gažová, N., & Jirásek, M. (2016). Framing and bias: A literature review of recent findings. Central European Journal of Management, 3(2), 23-32. https://doi.org/10.5817/CEJM2016-2-2

Janiszewski, C., Silk, T., & Cooke, Alan D. J. (2003). Different scales for different frames: The role of subjective scales and experience in explaining attribute-framing effects. Journal of Consumer Research, 30(3), 311–325.