Join our Facebook Group where you can share, talk and find updates about CRO
Join our Facebook Group
Posted on Mar 14, 2020
The corona virus has an increasingly visible impack on e-commerce, especially when dealing with something like plane tickets. This made me think about the impact this could have on A/B tests. To be able to test properly you need a sample size that's representative of all of your visitors.
Imagine if the sample size is not the problem, because you're certain you can fulfill that requirement. Have you thought about how the visitors in your current tests are feeling at the moment? Could these feelings have any effect on your test results? These feelings are very likely to be temporary, so the results from the tests might not be applicable for long term usage.
What I'm trying to tell you is that you should be aware of this situation, because the test results you receive could still be "false positive" results.
I'm very interested to find out what your opinions are on this and how you deal with this situation.