A/B and MVT Testing

A/B testing (sometimes called split testing) is comparing two versions of a web page to see which one performs better. You compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. The one that gives a better conversion rate, wins!

 

All websites on the web have a goal - a reason for them to exist

 

  • eCommerce websites want visitors buying products

  • SaaS web apps want visitors signing up for a trial and converting to paid visitors

  • News and media websites want readers to click on ads or sign up for paid subscriptions

 

Every business website wants visitors converting from just visitors to something else. The rate at which a website is able to do this is its "conversion rate". Measuring the performance of a variation (A or B) means measuring the rate at which it converts visitors to goal achievers.


 

Why A/B Testing?

A/B testing allows you to make more out of your existing traffic. While the cost of acquiring paid traffic can be huge, the cost of increasing your conversions is minimal. To illustrate this, an example of our “small business plan of GemSites Optimiser starts at £35. That's the cost of 5 to 10 Google Adwords clicks. The Return On Investment of A/B testing can be massive, as even small changes on a landing page or website can result in significant increases in leads generated, sales and revenue.

The A/B Testing Process

The correct way to run an AB testing experiment (or any other experiment for that matter) is to follow the Scientific Method. The steps of the Scientific Method are:

  • Ask a question: "Why is the bounce rate of my website higher than industry standard?"

  • Do background research: Understand your visitors' behaviour using Google Analytics and any other analytics tools running on your website.

  • Construct a hypothesis: "Adding more links in the footer will reduce the bounce rate".

  • Calculate the number of visitors/days you need to run the test for:Always calculate the number of visitors required for a test before starting the test. You can use our A/B Test Duration Calculator.

  • Test your hypothesis: You create a site wide A/B test in which the variation (version B) has a footer with more links. You test it against the original and measure bounce rate.

  • Analyze data and draw conclusions: If the footer with more links reduces bounce rate, then you can conclude that increased number of links in the footer is one of the factors that reduces bounce. If there is no difference in bounce, then go back to step 3 and construct a new hypothesis.

  • Report results to all concerned: Let others in Marketing, IT and UI/UX know of the test results and insights generated.