The ABC's of A/B Testing (and some optimization too)

Crazy Little Thing Called "Optimization"

Optimization. The epitome of stereotypical business jargon (aside from "synergy"). But what does it really mean, and how is it more than just a fancy word to say to your friends at cocktail parties? According to Business Dictionary, optimization is defined as "Finding an alternative with the most cost effective or highest achievable performance under the given constraints, by maximizing desired factors and minimizing undesired ones". Optimization can be applied to a variety of working parts, whether it's search engine optimization, web optimization, or content such a images, videos, or news.

The benefits of optimization? Endless. But IBM mentions the top few in their article, "How companies use optimization". Those of which are that optimization maximizes operational efficiency, uncovers solutions to challenges, creates a ROI that is measurable, and is a concept that is important and applicable to every industry.

Tests! Aah!

Experiments: applicable in more than just science class. A/B testing brings experiments and hypotheses to the marketing world, by offering companies the advantage of knowing what techniques will work for their websites and apps before permanently implementing them, or making changes that might be unnecessary.

Essentially, A/B testing is a process by which two versions of a website or app are compared against one another to see which one does a better job of reaching customers on a variety of dimensions. These dimensions include micro-conversions or macro-conversions, and how different aspects of a website can effect these different events. A/B testing is implemented by companies first creating a webpage or app, and then duplicating it, and modifying it, to create a second "test" version that only differs from the original in a small variation. Only changing one piece of the web page lets marketers know that if there is a change, that chances are its due to that single piece. Otherwise, like in science class, if there are too many variations in an experiment, its going to be difficult to determine where the effected results are coming from. After duplicating the web page, both are made live and half of the traffic received views the original, unmodified version, and the other half are shown the modified version. In this case, the original is the control and the modified version is the variant (yay, more science stuff). As different visitors engage with the web pages, the data collected is gathered into an analytics dashboard and is put through statistical tests to come up with information that is useful for companies to determine what changes on their websites are successful (and profitable).

Wouldn't it be easier to simply mimic whatever the competition does on their website? Well, maybe. But would you also mimic their product to a T? Probably not. Just because a competitor is a competitor doesn't mean that you don't have a different audience whose media needs need to be met in a different way. Through using A/B testing, companies are able to decide relatively risk-free what web content is ideal for their marketing and their consumers. And after all, maybe these differences in the web content are what will drive the competition's consumers to switch over to becoming your consumers.

EA: The A/B Test Golden Child

Still hesitant about implementing A/B testing? Well, Hubspot created a list of three major companies that have success stories with this online testing. Let's take a look at one of those three; EA. Electronic Arts had a simple goal: increase revenue. I'll get straight to the point, the outcome of the decision that was driven by the A/B test resulted in 1.1 million copies of Sim City being sold within the first two weeks of its release. 50% of the sales were generated via online eCommerce. The original web page that they were testing was a page for the Sim City game, with a giant banner at the top that offered a promotional deal for pre-ordering the game, with the initial idea that this would drive sales. However, the company decided to test a simpler web page that did not include the promotional banner.

To the company's surprise, and my own after reading the article, "The variation with no offer messaging whatsoever drove 43.4% more purchases". This is surprising because you would think that consumers would love a monetary reward for pre-ordering a product. However, in EA's case, promotional tools did not drive sales as much as a simple web page did. This is a great example of the power of the A/B test, because without it EA would have had no idea that their customers were interested in a lack of banner ad offers.

A/B Testing: Are you Convinced Yet?

From Forrester's "Optimize Customer Experiences with Online Testing and Continuous Optimization" the graphic below depicts a pie chart of a breakdown of businesses that use online testing. 14% believe that their digital analytics program is not integrated at all, 55% believe its somewhat integrated, and 31% say analytics is integrated. The following graphic displays the same businesses and their thoughts on the benefits that they've received from online testing. When it comes to increased conversions, all businesses, regardless of their digital analytics program strength could see the positive effects of A/B testing. This speaks volumes to the importance that online testing represents.

And if all else fails, its important to note that according to Forrester, 61% of firms have had an active internet testing service in place for the last two years. So if your business is wanting to take the next step up the marketing and optimization ladder, time to join the club.