Home Theories Survivorship bias

Survivorship bias

Reviewed by expert Scientifically proven

Sure! Survivorship bias is a cognitive bias where we tend to focus on the things or people that have survived a particular process or event, rather than on those that did not. In other words, we tend to overestimate the importance of certain factors in success, simply because we only look at those who have succeeded, and we ignore the role of chance or other external factors that may have contributed to success or failure. This bias can lead to flawed decision-making, as we may underestimate the risks or overestimate the opportunities of a particular situation based on incomplete or biased information.

Table of contents:
  1. What is Survivorship Bias?
  2. Examples of Survivorship Bias in CRO
    1. Example #1: Testimonial Bias
    2. Example #2: A/B Testing Bias
    3. Example #3: Traffic Source Bias
  3. How to Avoid Survivorship Bias in CRO
    1. Strategy #1: Collect Both Positive and Negative Feedback
    2. Strategy #2: Test More Than Two Variations
    3. Strategy #3: Analyze All Traffic Sources
  4. Conclusion

Sure, here's a blog post about the survivorship bias:

Understanding Survivorship Bias in Conversion Rate Optimization

Survivorship bias is a cognitive bias that can significantly affect conversion rate optimization strategies. This bias happens when you only consider the successful cases and ignore or overlook the failed attempts, leading you to arrive at an incomplete or inaccurate conclusion.

In conversion rate optimization, this bias can lead to wrong assumptions about what works and what doesn't. To understand this better, let's take a closer look at what survivorship bias is and how it can impact your approach to CRO.

What is Survivorship Bias?

Survivorship bias, also known as "survival bias," refers to the tendency to focus on the successes, rather than a full range of outcomes. This bias typically happens when you only look at data or examples from the people or things that survived an experience while ignoring the ones that didn't.

For example, you might only look at successful case studies or testimonials and ignore the ones that didn't work. Or, you might only analyze data from customers who completed a purchase and not the ones who abandoned their cart.

When you ignore the "non-survivors," you can develop a skewed understanding of what makes for success. This is because you're only seeing the variables that work, not the variables that didn't.

This kind of bias can be particularly problematic in conversion rate optimization, where every piece of information is essential to making informed decisions.

Examples of Survivorship Bias in CRO

To illustrate the impact of survivorship bias on conversion rate optimization, here are some examples of how it can impact your strategies:

Example #1: Testimonial Bias

Testimonials can be a powerful tool for persuasion, but they can also be subject to survivorship bias. If you only use testimonials from customers who had an excellent experience with your product or service, you might be missing out on valuable feedback from the customers who didn't.

By ignoring negative feedback, you're missing out on opportunities to improve your product or service, which could lead to a higher conversion rate down the line.

Example #2: A/B Testing Bias

A/B testing is a popular tool used in conversion rate optimization to test different versions of a page or element to see which one performs better. However, survivorship bias can creep in when you only test the versions that "survived" the testing process.

For example, suppose you're testing two different versions of your landing page. Version A performs better than Version B, and you decide to implement Version A on your website. But what if there were other versions that you didn't test that could have performed even better?

By only focusing on the successful test, you might miss out on valuable insights that could help you improve even further.

Example #3: Traffic Source Bias

When analyzing your website's traffic sources, you might be tempted to only focus on the ones that generate the most traffic. However, this type of analysis can be subject to survivorship bias because you're only looking at the traffic sources that "survived" the competition.

For example, you might focus solely on social media traffic and ignore other traffic sources such as email marketing or paid advertising. By ignoring these other sources of traffic, you might be missing out on potential opportunities to improve your conversion rate.

How to Avoid Survivorship Bias in CRO

Now that we've seen how survivorship bias can impact conversion rate optimization, let's look at some strategies to avoid it:

Strategy #1: Collect Both Positive and Negative Feedback

To avoid testimonial bias, collect both positive and negative feedback from your customers. By doing so, you'll get a more comprehensive understanding of the customer experience and identify areas for improvement in your products or services.

Strategy #2: Test More Than Two Variations

To avoid A/B testing bias, test more than two variations of your page or element. By doing so, you'll get a more comprehensive understanding of which elements work best and which don't.

Strategy #3: Analyze All Traffic Sources

To avoid traffic source bias, analyze all of your website's traffic sources, not just the ones that generate the most traffic. By doing so, you'll identify potential areas for improvement and opportunities to increase your conversion rate.

Conclusion

Survivorship bias can significantly impact conversion rate optimization strategies by leading to incomplete or inaccurate conclusions. To avoid this bias, it's essential to collect both positive and negative feedback, test more than two variations, and analyze all traffic sources.

By doing so, you'll get a better understanding of what works and what doesn't, leading to more informed decisions and a higher conversion rate in the long run.

A new cognitive bias in your inbox every week

You'll get to learn more about CRO and psychology. You'll be able to take experimenting to a whole new level!

* We send our mails on Monday morning btw.

Will you use psychology for your experimentation process?

Are you curious about how to apply this bias in experimentation? We've got that information available for you!

Join over 452+ users

  • Lifetime access to all biases
  • Filter on metrics, page type, implementation effort
  • More examples and code for experimentation

Choose your subscription!

Pay with Stripe

Lifetime deal PREMIUM

Get access to the search engine, filter page, and future features.

€499.00
I want this

Lifetime deal

Get access to the search engine and filter page.

€149.00
I want this

Do you think you know enough about CRO?

Join our monthly mailing to continue learning more and more about CRO and psychology.