What is the A/B test hypothesis?

We all know that A/B testing is about comparing the performance of a certain element (ex: a call to action) of a webpage of its current version against the modified version to determine which version acts better towards Conversion Rate Optimization (CRO) followed by more returns. However there’s a misconception about A/B testing as some have taken this mechanism quite wrong thinking that it’s all about simply changing a layout of a form, changing a label of CTA etc. But, it’s not that. And that’s when many fail to succeed with Conversion Rate optimization (CRO) even though they think that they changed everything for the best.

To reap the real benefits of A/B testing, recognizing the issues or problems in your design is as important as determining hypotheses that will bring you the best results. What is this so-called hypothesis? Well, to put in the simplest terms, hypothesis is the way you plan to make improvements to a selected element of a webpage.

Now here’s how you add hypotheses into the mix!

• First, you identify the key problem of your page. Let’s say there is a considerable high dropout rate on the checkout page.
• Secondly, you recognize the root cause for the problem. Let’s say that the root cause is the poor visibility of the subtotal of each item in the cart.
• Finally, you come up with the hypothesis precisely based on the root cause. How are you going to improve the design? Think about it. If you merely did some changes to the design after the identification of the problem without thinking through a proper hypothesis, then all your efforts would be in vain. Moreover, if you come up with a hypothesis by poorly recognizing the problems, then you’re in trouble again!

How to identify the problems clearly to have a right base for A/B testing hypothesis?

• Use web analytics data to determine your conversion problems.
• Use ergonomic audits as well as heuristic evaluation to evaluate the user experience generated by the web page.
• Perform user tests to have precise insight.
• Use heat maps to track the user interactivity amongst the elements on the same page.
• Use customer feedback from reviews, complaints etc.

Now that the problems are correctly identified, it’s time to initiate the formula of A/B test hypothesis. Let’s assume that the checked element is ‘A’, its present status is ‘A1’ and the proposed change is ‘A2’.

Hypothesis = changing A from A1 to A2 will create an impact of X (X is the result of the change)

Quite simple, isn’t it? But the key here is that the impact (X) should be measurable in terms such as conversion rate, checkout rate etc. in order to determine the real success of your hypothesis. That is the core point you should think about when determining the hypothesis; how am I going to measure the results of my action plan?

Let’s dig into some examples of hypotheses taken from real world e-commerce scenarios

1.      Zalora.com changed their labels next to each product from ‘Delivery above $150 and 30 days return’ to ‘Free delivery above $150 and Free 30 days return’ to create an impact on checkout rate increment.

2.      Corporate.payu.com changed their check out page from asking both email address and the mobile number only to ask the mobile number of the user to create an impact on increased conversion rate.

Best proven practices of A/B test hypothesis in e-commerce towards Conversion Rate Optimization (CRO)

• In order to extend visitors’ stay, have a banner in the homepage including products or services that you offer in order to increase the curiosity factor of the visitors.
• Insert a CTA in the landing page as there’s a high possibility that your visitors may click on it because they are new and they want to know more immediately.
• Embed a filter to the product category page in order to save time.
• Highlight the mostly bought products in the product category page to give the visitors a kick start.
• Make sure that ‘add to cart’ button is clearly visible at the first glance in the product page.
• Make sure that ‘proceed to payment’ button is clearly visible at the first glance in the cart page.
• Try to keep only one payment page for payments to reduce time of checking out.

Conclusion

Having a hypothesis is a key element to have success in A/B testing towards Conversion Rate Optimization (CRO). Moreover, the impact generated by the hypothesis should be measurable to make the most of your effort of A/B testing. When A/B testing data is put into use by adhering to the best practices we discussed throughout the article, you can drive the success of A/B testing effort towards Conversion Rate Optimization (CRO) quite effectively!