A/B Testing and Conversion Rates

When I think of running experiments, I picture people in white coats in a laboratory with test tubes. However, experiments have an important place at the heart of marketing. Market research is a huge part of how we learn about consumer behavior and responses. A more specific form of experiments in marketing is A/B testing. In this blog, I'll be discussing the basics of A/B testing and conversions.

What is A/B Testing?

A/B testing is the process of testing a control and a variation of an ad, web page, or email to see if consumers better respond to a variation of a certain digital element. A/B testing is closely related to UX design, as a variation can be a complete redesign of a page, or as simple as moving a button by a few pixels. During the A/B test, the control page and variation page are randomly distributed to visitors. After a certain amount of time, the results are statistically analyzed to determine if the variation has a more resonant effect on visitors.

The process of A/B testing involves 6 main steps:

1) Collect data to understand which parts of your site are falling short.

2) Identify which conversions your company needs to optimize.

3) Create an informed hypothesis based on consumer knowledge of what will garner a better response than the current control model.

4) Create the variation for the experiment using UX design.

5) Run the A/B experiment for a designated amount of time.

6) Collect and analyze data to see if there is a statistically significant difference between the control and variation.

Why A/B Testing?

A/B testing allows marketers to get glimpses into what consumers respond to in a quantifiable way. When it comes to digital marketing, small changes to a landing page can make a big difference, but consumers might not understand themselves why this is. In the words of Steve Jobs, "A lot of times, people don't know what they want until you show it to them.". A/B testing gives marketers a way of showing consumers two different versions of what they might want, and measure the success of each.

The ultimate goal of A/B testing is to increase the amount of conversions on a site. Simply put, conversions are the processes of consumers doing what you want them to do. A conversion could be a click, a purchase, or any other action that marketers aim to have consumers take. The conversion rate for a site can be calculated by dividing the number of conversions by the amount of site traffic x 100. This produces percentages that can be compared for both the control and variation in an A/B test. Conversion rates enable marketers to simply quantify and compare the two groups in an A/B test and run statistical analyses to determine if there is a statistically significant difference between the two.

Case Study: Groove HQ

The process and success of A/B testing can be exemplified by the experience of GrooveHQ, an SaaS provider. Groove knew that they needed to update their landing page, and they knew that what they were doing was not working. A/B testing seemed like the answer, but it costs thousands of dollars. By going through a detailed and informed research process before A/B testing (basically, extending step #3 in the process I detailed above), Groove was able to better understand their consumers and optimize their conversion rates.

Groove started their research process by first increasing their knowledge on CRO and A/B testing from free sources. Getting up to speed with the processes involved in the experiment would ensure that they were getting the most out of every step of the way. Next, Groove used their connections to their advantage. They asked experts in the industry look at their landing page and point out the holes that might be misleading to visitors. Finally, Groove came to both new and old customers to honestly ask what they would like to see improved and understand the language that they use to talk about the product.

This process, though seemingly long and tedious, is a very important part of A/B testing. Although UX and design is crucial to building a great landing page, it can't be effective unless the company has a clear idea of what will resonate with their customers. This approach was highly successful for Groove, as conversion rates went from 2.3% to 4.3% when the variation was implemented. By taking the long route to A/B testing, Groove found success through CRO and a greater knowledge of customer problems that they aim to solve.