A/B Testing

• Let's provide A1 milk to 50 children and A2 milk to another 50 children, then observe their average weight after one year.
• Place the "30% discount" text as the first word in the advertisement and let's test this new page on 10% of customer traffic.
• Team!!! Move the recommendation to the top of the page instead of the bottom, and divert 20% of the traffic to the new page.

What are we doing here? We are comparing two versions of an app, webpage, or medicine to identify which one performs better. This is what A/B testing is. With A/B testing, we don't just guess behaviour; instead, we make decisions based on real data.

Why do we do A/B testing?

Data Drive Decision

This approach helps in understanding customer behaviour and preferences, enabling more informed and accurate decision-making. Instead of relying on guesswork

Increase Conversion Rate

This is obviously the primary objective. With continuous A/B testing companies fine-tune the user experience , that ultimately leads to higher engagement and increased revenue.

Low Risk

Test is performed on fraction of customers, and if anything goes wrong then quickly divert the traffic to existing page. Isn't it very safe and effective way to innovate and improve?

So how do we do the A/B testing?

Set the Goal

Identify what you want to test? Determine what metric you will measure to evaluate success, like conversion rate or user engagement.

Define Hypothesis

Develop hypothesis about what would you like to change in order to achieve the goal? For ex: Placing the "30% discount" text as the first word in the advertisement will increase the clicks."

Define Control & Treatment

1. What's test? Create 2 groups: A group with existing behaviour and B Groups is modified version with the changes.
2. Run the test: Once all the setup is done, then start the A/B test for sufficient time to gather meaningful result.
3. Conclusion: Determine who the hero is. If B (Treatment) has performed better, we can conclude that the changes are positive.