A/B testing is an incredible tool to have in your marketing arsenal. By definition, A/B testing allows you to introduce a change in one variable of your site. This post elaborates on what that variable can be, this also dictates the maturity of your A/B testing efforts.
Versions of Content
Basically wherever you have versions of content (Audio, Images, Text, Video etc.) you can use A/B testing. Now the underlying versions might be different in one way or the other. Some of the different versions can be :
- Anything related to fonts : Font size, Font Face etc.
- Anything related to color
- Anything related to size
- Anything related to look & feel / Completely different content
These form the most basic of A/B tests and are widely used. However, in no way are these A/B tests inferior or yield less business results. These tests are especially helpful when you are discussing which banner/CTA/Headline/Promotional Offer/Size of product pack /product description is better.
Location of content
In this type of A/B test – you want to change just the location of the content.
Moving that “contact us” from the left side to right side is an example of this test.
Moving a menu item from one place on the header to another
Basically, you want to play with the location of your content. The results can be surprising.
Number of fields in a content object
This is applicable when you have a content object. Survey & Registration Forms are good examples of content objects. Here we want to try the ideal number of individual units in a content object. For example , a registration form with 3 fields may be better than 5 fields. This is a simple scenario & answer is self explanatory. However, there are complex scenarios – the answers to them are only obtained through A/B testing.
These tests are in no way limited to forms – you can use these to experiment whatever you want to experiment on. Ex: Identify the ideal number of reviews to be put on a page during checkout site. We frame hypothesis and test it using A/B testing.
Here we just experiment with the page layout. Examples of these tests are :
- Number of content columns / tiles
- Size of the content columns etc.
In this case, our variable is the user flow. Let me give you a quick example. An eCommerce site has below journey for check out
Check out – > Enter Shipping Information – > Enter Payment Details -> Order Confirmation
There can be so many ways to build this journey:
Check out -> Enter Payment Details with check box for using earlier used Shipping Address -> Order Confirmation
I guess this explains what we are trying to do here.
If there are different algorithms to determine the price on site and you want to see which algorithm fares better – then we run a A/B test to find the winner.
If there is a new feature going live – we test it to see how it resonates with the users. We build a hypothesis and that dictates the test. Example:
- Does auto play of video on product display lead to more conversions ?
- Does auto populating the payment information with a placeholder to enter CVV lead to more conversions?
- Does adding a visual indicator of shopping cart progress lead to more conversions ?
- Does adding new sections on pages (reviews, number of purchases etc.) lead to more conversions
- Will prompting for registration later on lead to more conversions?
So, in this way – we start with basic A/B testing and the move on to use A/B testing for tactical decision making. A/B testing is both art and science. Art is designing the test while implementing it is science.
Adobe Target , Optimizely & VWO helps in accomplishing few of these scenarios quite easily, but it requires a lot of commitment to implement such A/B tests for continuous decision making.
Note : There are some nuances of A/B testing that one should be aware of, these are often ignored. The below article captures these very well.