Julie Boisen Schmidt

Performance Marketing Executive

June 4, 2020

The Benefits of AB Testing

Introduction

You’ve probably heard about the marvellous things that A/B testing can do for you. With it, you’ll probably never have to worry about decision making anymore. Because it doesn’t matter if your ad creatives are in yellow or blue or whether your website holds a more minimalist style or otherwise, as long as you have a solid answer what your customers are into – you’re already on your way to achieving your marketing goals. 

It’s no surprise that many companies understand the importance of data in today’s world. It doesn’t matter what field you’re in or what product or services you sell, what really matters is the content you put out and whether or not it is relevant to your viewers. And what better way to find out other than consistently conducting A/B tests. Hence, throw away the mindset that an A/B test is just a “when I feel like it” tool when really, it’s more of a necessity. 

But wait for a second, what is A/B testing?

What is A/B testing?

A/B testing, also known as split testing, is a method of comparing two versions of the same digital marketing assets against each other to test which out of the two performs better. You can use A/B testing for your webpage, Facebook Ads, emails, and more!

What’s so great about this test is that it enables you to understand what works and what doesn’t.

How does an A/B test work

To understand how A/B testing works, let’s take a look at the example below. 

For instance, your goal is to get more newsletter sign-ups on your website. Your current web design (in this case, to be referred to as “the controlled”) is performing decently, but you want to test if your new landing page (to be referred to as “the variant”) could get you more leads. Instead of jumping into a conclusion which landing page would perform better, you run an A/B test. 

You show Website A (old) to one group and Website B(new) to another group. Remember to be patient as it takes time to run this test. Once you’ve seen the results, you can make a sound decision to either keep the old website or implement the new one. This way, you don’t discard what’s already been working nor do you miss out on something that could bring you better results! In short, A/B testing is the best way to eliminate “we think this would work” and have a solid “we know this works”. 

However, keep this in mind that whenever you perform an A/B test you should only test one element at a time or else you would have a hard time figuring out what defines a successful test. For instance, it is easy to know what causes Banner A to perform better then Banner B if the only thing you’re testing is the ad copy. On the other hand, if you’ve written a different ad copy and have a different background colour for both ads you’re testing, you won’t be able to tell if it’s the copy or the background colour that had made the viewer click on the ad.

Benefits of A/B testing

Now that you have an idea what A/B testing is and how it works, we’re going to step into the benefits of how A/B testing can impact and change your marketing strategy. Let’s dive into it.

1. Improve your content

The phrase ‘Content is King’ still has its truth. Think about it, would you click on something that you can’t relate to? As a marketer you want to generate contents that are of value to your viewers, they have to like what they see in order to engage with your ad, website, or email. This is when the nifty A/B testing tool becomes your best friend. When you A/B test your marketing creatives, it gives you a chance to filter out and improve ineffective contents. The winning result often gives you a better idea of what your audiences like or found relevant. With that information, you’re able to generate better, more relatable content that will help build credibility and trust for your brand.

b. Reduce bounce rate

If you own a website, bounce rates are your worst enemy. Bounce rate is the percentage of people who visits your website and leaves without viewing other pages within the website or leaves within a very short time span. If your bounce rate is high it means that viewers often do not stay on your website for long. That’s no good for business because you want them to learn more about your brand and hopefully leave an email or buy a product or service. With A/B testing you’re able to test a newer version of your landing page against the current one to find which version gets you a higher engagement rate that encourages users to browse longer on your website.

c. Increase conversion rate

Say you have a conversion campaign target to hit – you’ll obviously need an ad that works in order to entice customers to buy your product or services. So how do you know what type of ad creatives or copy work? You’ve guessed it right – conduct an A/B testing. Although it may sound like there’s more work to do by creating a variant, the better question is: “Would you rather accept the current state of performance, or try to find ways to improve it?”

d. Easy to analyze

In order to achieve something, you first need to know what you want to achieve. Setting a goal before getting started with anything is key to a successful campaign. It sets a clearer path for you to lay out your marketing strategy whether it’s copywriting, running a test, or building a landing page. Similarly, to A/B testing, when you have a clear goal to accomplish analysing the test report would be relatively easy as you’ll know what metrics to look at (conversion, impression, etc).  For instance, if your goal was to achieve more engagement through your Facebook ad — clicks, impressions, and post engagement are the metrics you’ll be looking at when the report is ready. And whichever ad creative with the highest goal value, that’s the new ad creative direction you want to push for in the future.

e. Quick results

Another perk of A/B testing is that it can give you quick results on what is working and what is not. Even if you’re testing on a relatively small sample size, an A/B test is able to provide actionable results that can help you make quick and accurate decisions – allowing you to free up your time to focus on other matters that are important to your business.

f. Multi-functional

There are no bad ideas when it comes to A/B testing as it is not biased and does not have a ‘preference’. No one is there to physically tell you that your ideas don’t work as you will be able to base your decisions on collected data. Not to mention that there are so many parameters that you can test – from colours, fonts, ad copies, images, any element you can think of. Besides testing the ad or website content, another type of A/B test is audience testing. Here’s how; you could show the same ad or website version (anything that you want to present) to two different audiences. For instance, Group A consists of people between age 18-25 whereas Group B consists of people age 26-30. This is one way you can understand what appeals more to the audience you want to target. Once the test is over, you’ll realise that one group performs better compared to the other. Perhaps they show more engagement or there’s more conversion from a particular group. Whatever your goal is, the group that helps you achieve it is the one that you want to target.

g. Reduce risks

Even if you’re planning to conduct a very well thought out change in any of your marketing creatives, there’s still a chance that you might be sabotaging your own ad campaign, email send-outs, or even traffic on your website. This is because you can’t determine what your viewers like or dislike. As a result, this can, in the end, affect your sales if your audiences cannot adapt, relate, or simply do not like the new version of your creative assets. However, if you perform an A/B test, you are sure to find out if something works or not. This minimises the discussion of “will this work?” and the back and forth of creating new versions of creatives blindly. 

Set up your A/B test in 10 steps

There are 3 stages of an A/B test, which are, before, during, and after. Let us quickly go through each stage so you can start testing your ads!

Before your A/B test

1. Identify your goal

  • Before running a test, make sure you know what your intentions are. What’s the purpose of this test? Is it to look at conversion rates, impressions, etc. A clearly defined goal will help you in determining whether this test is successful, or not.

 

2. Pick one variable to test

  • While it’s great that you’re excited to test all sorts of variables, it’s best practice to test a single variable at a time. Otherwise, it will be hard for you to understand the exact element that causes a change in performance.

 

3. Create two versions of your ad (Control and Variation)

  • Before you run your test, you should create two versions of your ad. Essentially these two ads are similar except for the variant you would like to test. See example below:

4. Determine your audience

  • We want to test these creatives to people who are active or people who are more likely to engage with your post so that we can see results between the two creatives. Hence, it would be smart to pick an audience that you’ve tested on before. Reason being, you know that these audiences are a group of people who are more likely to respond to an ad from you. However, this step is of course not applicable if you are using the A/B test to find the best performing audience.

5. Decide how significant your results need to be

  • Before starting your test it is important to establish how significant your results need to be to justify choosing one ad over another. For instance, if you have only made a very small change e.g. changed the colour of a button, you have to figure out how big of a difference between the two variants you will allow before determining a winner of the test. If colour A gets 15 conversions, while colour B gets 17 conversions will this be enough of a difference to determine a winner? To determine how significant your result should be you have to look at statistical significance. Your results are considered “significant” when they are very unlikely to have occurred by chance. Achieving statistical significance with a 95% confidence level means that your results will only occur by chance once in every 20 times. You can calculate your statistical significance by using this calculator.

During your A/B test

6. A/B testing tool 

  • To create an A/B test you need an A/B testing tool. Both Google and Facebook have their own A/B testing tool where you can measure the results from your ads. 

 

7. Test your ads at the same time 

  • To limit out any outside factors that can change your results, it is important that you test your ads at the exact same time. By doing that, you are certain that the difference in the result is not due to any factors besides the changes in your ad. 

 

8. Make sure the test runs for a sufficient amount of time 

  • Based on how significant you want your results to be, you can set a timeframe for your tests. This will give you enough data to analyze afterwards. For instance, if you’re running an A/B test on a website, your test depends on how many visitors get per day. Say if there’s an average of 100 people that are visiting your website a day and your goal is to at least have 500 sign-ups, then the duration of your test depends on how quickly these visitors sign-up. Say if you’re lucky and 100 people sign-up each day, that would mean that your test duration lasted for 5 days. There are different ways to calculate it, but one way could be to use a calculator.

After your A/B test

9. Analyze your results

  • Although there are multiple metrics to look at in an A/B test report. It’s important to only focus on those that make sense to your goal. For instance, if your goal is to achieve a high conversion rate, then it’s a no brainer to look at CVR (conversion rate ). On the other hand, if you’re looking to gain more engagement, then you should be looking at impressions and clicks. Remember, focus on your goals!

 

10. Next steps 

  • Now that you have tested and have a winner marketing asset, it now gives you an idea of what your direction should be moving forward. However, keep in mind that you always have room for improvement so keep on testing your creatives! 

Conclusion

Remember, A/B testing is one of the many powerful ways to collect information from our customers. It helps us make better decisions and to also provide more relevant content to our viewers. Keep in mind that performing a single A/B testing will not seal the deal. You should always make it a habit as a marketer to consistently perform these tests so that you’re able to optimise your marketing creatives.

Good luck!

If you’re interested and ready to improve your creative performance through A/B testing, you can drop us a note at [email protected]. We’re more than excited to assist in your brand growth! 

Julie Boisen Schmidt

Performance Marketing Executive

Share this post!

Share on facebook
Share on twitter
Share on linkedin

Other posts that you might interested in

Is Your Facebook Campaign Under Performing? Here Are 6 Possible Reasons.

What Is Attribution and Why Does It Matter?

How to Preview What Your Website Looks Like on Social Media