A/B Testing. Process and Analysis
To win new customers and remain relevant, a business needs to evolve. Without the introduction of new ideas, projects become boring and ineffective. Website redesign, changing creatives or calls to action in advertising, expanding the range, introducing promotions or discounts — how to understand what will bring the result?
One of the types of marketing research is A/B testing. This method allows you to test hypotheses and analyze the reaction of consumers to changes.
A/B testing (or split testing) is a marketing research method where the essence of an experiment is that a control group of items is compared with a set of test groups in which one or more items have been changed to find out which changes improve the target indicator. Thus, when testing, option “A” and option “B” are compared, and the goal is to determine the best of the two tested options.
A/B testing in contextual advertising
In contextual advertising, A/B testing is used to analyze headlines, descriptions, visual elements such as banners or videos. In a study, the experimental element “B” is compared with respect to the main setting “A”. That is, you can check two versions of the same object, with different elements, other things being equal in the conditions. The purpose is to find out which elements increase efficiency.
Split testing is also used to analyze the various components of an ad:
- Advertisements. Test different variations of titles or descriptions: a call to action, an ad with or without a price, etc.
- Bidding strategies. Usually, smart bidding strategies are the most effective but, in some cases, manual bidding strategies work better.
- Landing pages. Check the options for placing buttons or products on the site, their visual design.
- Banners. This case has almost no restrictions. Try experimenting with visual content, colors, text components, buttons.
- Video. Check out videos of various duration and content.
- Additional links. Add additional links to various sections of the website to your ad, e.g. blog, contact information, product categories, promotions page, and compare how the CTR will change.
- Refinement. Expand your ads and point out competitive advantages that are not included in the headings and descriptions.
- Final URL. The landing page address may not match the link that the user sees in the ad. This helps convey information about where the visitor will go by clicking on the ad. Therefore, you can test different link options and see which one works best.
Almost any ad can be improved; often already working texts can no longer catch the audience. The A/B testing method allows you to increase the effectiveness of advertising campaigns with no need to expand your budgets.
A/B testing example
Goal: Increase CTR
Hypothesis: Changing the call to action will increase the CTR
Element: Call to action
For the current campaign in Google Ads, we are launching an experiment with a modified call to action in the ad text for 50% of the daily budget, duration — 1 week (or until enough statistics are collected). Once the experiment is completed, we can conclude which ad was more clickable.
A/B testing evaluation metrics
Usually, there are 5 main metrics for evaluating A/B testing in contextual advertising.
- CTR. Increasing this indicator affects the quality of the ad and, as a result, reduces the cost per click.
- Conversion rate. Shows the way the target audience was attracted to the site. The disadvantage of evaluating this metric is that during the conversion process, the user is influenced by various factors (product price, website) that are beyond the responsibility of a contextual advertising specialist.
- CPA. This can be guided by if we understand the marginality of the product and can determine the threshold cost of a conversion, at which the conversion will be economically beneficial for the business.
- ROAS. This indicator is used in e-commerce projects to compare ad costs and product revenue.
- Impression-based metrics (CPI, RPI). They help to analyze the impact of displaying ads on the further interaction of users with the site.
A/B testing should be ongoing and regular. There are no perfect ads, landing pages and strategies. It is necessary to constantly generate new theories and hypotheses, look for ways to improve efficiency and test changes.
Split testing in contextual advertising
- The experiment is designed separately for each ad rather than for the entire ad group. This is necessary to understand which element influenced the test result.
- Duration of the experiment. The longer the test duration, the more reliable the result. This is due to a change in demand for most goods and services, which may depend on seasonality, changes in the exchange rate, market reactions to certain significant events. So, the experiment should continue until a clear “winner” is determined.
- Heterogeneous audience. Target groups show activity heterogeneously in different periods of time. Therefore, this factor also increases the time of the experiment.
- Lack of statistical data for low-frequency queries. To analyze the experiment, a relevant number of clicks on an advertisement is needed. But some low-frequency queries may only get a few impressions per week. In this case, the duration of the experiment can increase up to several months.
How to A/B test an ad?
Stages of the experiment:
- Formation of a hypothesis and a goal.
- KPI for the test result.
- Launching and conducting the experiment.
- Data analysis.
- Implementation of changes.
A/B test launch algorithm in Google Ads:
- Go to the “Experiments” tab in your ad account.
- Create a new experiment.
- Select a campaign, make changes according to the hypothesis, specify the name of the experiment, duration and budget distribution between the basic and experimental settings.
- Launch the experiment.
- Wait for completion or sufficient data collection.
- Analyze the data.
- Apply changes upon successful experiment. The system will prompt you to make new settings for the current campaign or start a new one.
The resulting statistics can be used not only in contextual advertising, but also in other areas of Internet marketing: texts of email newsletters, SEO metadata, and landing pages. Regular tests and experiments help adjust your entire marketing strategy.
A/B test quality checklist
- Ad impression is adjusted uniformly.
- The sample of the target audience, as well as data for analysis are large enough.
- The conditions for testing are the same for basic and experimental settings. You should not run the experiment during seasonal fluctuations in consumer activity.
- Only one item is tested. This is the only way to find out which change affected the result.
- Each test has a clear goal and hypothesis.
- The experiment should continue until enough statistics are collected.
If each element of the ad performs at its best, it will reduce ad costs and increase conversions. Our Well Web Marketing team conducts regular A/B testing on all projects. If you need our advice or support in advertising activities, please contact us. We are looking forward to further cooperation!