The Impact and Methodology Behind A – B Testing

by | May 17, 2018 | News | 0 comments

The behaviour of Internet users has evolved over recent years. With strong competition and more apps available on the digital-marketplace, users are less faithful to a particular site or product. As marketing budgets are becoming tighter, teams need to take a scientific approach to optimise the user experience; as demonstrated by Google testing multiple themes on their iconic Maps app. This is where A – B Testing comes in.

When it comes to digital marketing, rather than making a final decision unilaterally on a cross section of users, A – B Testing, which is seeing continued growth as a digital market-research tool, seeks to determine the best approach from the data of multiple versions of web content. The marketer will mobilize their audience based upon:

  • Submitting several versions of the same project over a fixed period: 50% of visitors see version A, 50% see B. Only one factor changes.
  • Performing fine analysis to see which works best: more sales, clicks, registrations etc.

Accordingly, they will then be able to make objective and quantified marketing decisions. It is an infinite process, which allows continuous improvements to the functionalities of a site/product, by eliminating the “losing” versions.

Steps and Key Indicators

The first step of A – B Testing is to identify the indicators to measure, set up a strict process and a defined period. Here are some concrete examples:

  • Test on the layout of a call-to-action button on a merchant site to improve the percentage of Internet users making an online purchase. Bounce rates and conversion rates will then be scrupulously studied.
  • Test the colour of a banner or artwork used to optimize click rates on an online ad
  • Test on the time of sending a newsletter to reach the highest opening rates (or the punch line / subject of the mail)
  • Test on the layout of a landing page : for example with or without image, to see which one displays the most important conversion rate
  • Test for the optimal number of fields on a form
  • Test the title of an article that generates the most shares, etc.

The success of a test will depend on:

  • the choice of these indicators, which should not be too secondary. For example, if you change only the content of a section in the site this is unlikely to result in significant variation!
  • the size of the traffic on a site / or the mail database of a newsletter. If these numbers are too low, the A – B test is irrelevant.

Analyse, Activate and Measure

A – B Tests use various tools to analyze, activate and measure data. The object is to ensure business decisions are ultimately based upon reliable data. Such tools include:

  • Heat maps: These, and scroll-maps or click-maps etc. are tools which make it possible to better understand the behaviour of on-site visitors. They are a necessary technique when you want to perform an ergonomic audit to identify optimization points.
  • Widgets: these are customizable elements ready to be deployed on a site to set up its optimizations (banners, pop-ins, countdown banner, visitor counter, etc.).
  • Machine Learning : An essential technology, which identifies the most promising visitor segments of a site. That is to say, the visitors on whom the company should focus to conduct its optimization campaigns.

A – B tests, sometimes referred to as ‘Split Tests’ allow you to test a hypothesis that changing an element on your web site will convert into, for example, an increase in conversions. As outlined by Applause, in a recent report into continued experimentation and testing, “If the test is inconclusive, you may need to rework your hypothesis.”  If there is a clear winner, then the task is to report the insights generated to all concerned.

Need a marketing agency in High Wycombe? Give us a call today.