A/B Testing Breakdown

Last week, we went over the importance and potential benefits Google Analytics can provide online businesses. I used the beginner and advanced training courses to study for the GAIQ exam. The training process was thorough, and it ultimately paid off. I passed the qualification exam and earned my certification for Google Analytics. Learning the skills to use Google Analytics to analyze real data was an extremely worthwhile task, and I look forward to more certifications in the future. Google Analytics is a great tool used by marketers to analyze user data to maximize the potential of a website. This week I will be talking about another tool used to analyze the effectiveness of a website, A/B testing. 

What is A/B testing?

A/B testing is a digital marketing technique where a website is split into two variations and tested against each other to figure out the optimal design. Each visitor will see either design A or design B, and the design that leads to a higher conversion rate is chosen as the optimal design.

Why should we use A/B testing?

A/B testing is an effective strategy that is quite simple in nature. Being able to directly compare two similar options against each other is a straightforward way to determine the more effective option. Maximizing the websites conversion rate is much less expensive than spending money on promotions to acquire new customers. In addition to this, A/B testing can be used over and over again to constantly improve and adjust to new changes. It uses hard data to make design decisions that have already proven to be effective. Another huge advantage of A/B testing is that it can be used on any type of website design. A websites conversion rate can be applied as a metric to all types of businesses and websites. For example, an e-commerce website can use A/B testing to optimize user experience from product page to checkout, while a content driven website can optimize the subscription process.

Examples of effective A/B testing 

Electronic Arts, a video game company wanted to increase the number of pre-orders on an upcoming game called SimCity. The promotion was posted on the banner of their website, but it wasn’t drawing as much traffic as they had hoped for. In response to this they used A/B testing to figure out how to increase the pre-order rate. One of the variations completely removed the ad altogether. The results from this were extremely shocking, as this version without the ad drove a 43.4% increase in pre-order rates. Rational marketing beliefs would believe that a direct promotion would be the more effective strategy, but A/B testing demonstrated that this might not always be the case.

Another example of effective use of A/B testing is to study ComScore. Their goal was to increase conversion rates on leads by increasing the visibility of a quote from a respected client from Microsoft. They tried different layouts and designs, even adding the Microsoft logo to draw attention. After the experiment, there was a clear winner which led to a on a whopping 69% increase in conversion rates.

Best practices for A/B testing

In order to use A/B testing to its maximum potential, it is extremely important to have a structured process for determining the best course of action. The first step in the process is to define website goals and determine the key performance indicators. The next step is to collect data over a period of time and use analytics to determine what part of the website isn’t optimized to the full potential. After this, it is important to generate a hypothesis on why the users are behaving the way they are. Once a hypothesis is created, the next step is to create different variations of the website in areas where the conversion rate is less than desired with the use of A/B testing software. These different variations could be anything from changing the look of the title page, to making a button more prominent and colorful. Once the changes have been implemented for the different variation, run the experiment in comparison to controlled variable. The final step in the process is to analyze the data and determine the most optimal solution. This process can be repeated as many times as necessary to reach the ideal conversion rate. 

A/B testing is an extremely useful tool that can be used in all types of websites. So if you're looking to improve an aspect of your website, A/B testing can surely help you.