How to A/B test without an A/B Testing Platform

Do you want to run A/B tests but don't want to pay for and keep track of another platform? If so, you can leverage custom dimensions and explorations in your analytics platform to optimize conversions and complete tests. 

In this post, I'll share an example of how to run and report on an A/B test in Google Analytics 4. 

As a quick refresher, in an A/B test, we configure two versions of a user experience, version A (control) and version B (variation). After we create these variations, users are split into one of the groups, and we measure the success of version A compared to version B. If the number of users who complete a conversion action in one variation group is statistically significant, then that variation wins, and we can adjust the user experience accordingly. This process is streamlined in A/B testing platforms, but you don't need to use one to complete an A/B test. 

Enter Google Analytics 4. 

To run a simple A/B test, define a new parameter to send on the user action. For example, if I want to optimize a form submission event, I'll send a new parameter called "experiment_variation" on form_submit that will contain the value of "form_submit control" or "form_submit button variation."

Here's what that dataLayer code might look like: 

window.dataLayer = window.dataLayer || [];
dataLayer.push({
 'event': 'form_submit',
 'form_name': <form name>,
 'click_text': <click text>,
 'experiment_variation': <experiment variation value>
});        

Now that I'm capturing experiment information on form_submit, I have a couple of options for reporting. 

• I can create two new form submit events, one for each variation, by using the "create event" feature and experiment_variation value in GA4. Then, I can mark these events as conversions and compare the metrics in GA4's conversion reports. 
• I can create a custom exploration in GA4 that compares the number of form_submit event actions that have the first experiment_variation value versus the second. 

Based on these findings, I can update the user experience when the variation has statistical significance. I can follow this process for other user actions and conversion events or update the parameter value for future tests on the same event.