A/B testing is an incredible tool to have in your marketing arsenal. By definition, A/B testing allows you to introduce a change in one variable of your site. This post elaborates on what that variable can be, this also dictates the maturity of your A/B testing efforts.
Versions of Content
Basically wherever you have versions of content (Audio, Images, Text, Video etc.) you can use A/B testing. Now the underlying versions might be different in one way or the other. Some of the different versions can be :
- Anything related to fonts : Font size, Font Face etc.
- Anything related to color
- Anything related to size
- Anything related to look & feel / Completely different content
These form the most basic of A/B tests and are widely used. However, in no way are these A/B tests inferior or yield less business results. These tests are especially helpful when you are discussing which banner/CTA/Headline/Promotional Offer/Size of product pack /product description is better.
Location of content
In this type of A/B test – you want to change just the location of the content.
Moving that “contact us” from the left side to right side is an example of this test.
Moving a menu item from one place on the header to another
Basically, you want to play with the location of your content. The results can be surprising.
Number of fields in a content object
This is applicable when you have a content object. Survey & Registration Forms are good examples of content objects. Here we want to try the ideal number of individual units in a content object. For example , a registration form with 3 fields may be better than 5 fields. This is a simple scenario & answer is self explanatory. However, there are complex scenarios – the answers to them are only obtained through A/B testing.
These tests are in no way limited to forms – you can use these to experiment whatever you want to experiment on. Ex: Identify the ideal number of reviews to be put on a page during checkout site. We frame hypothesis and test it using A/B testing.
Page Layout
Here we just experiment with the page layout. Examples of these tests are :
- Number of content columns / tiles
- Size of the content columns etc.
User Flow
In this case, our variable is the user flow. Let me give you a quick example. An eCommerce site has below journey for check out
Check out – > Enter Shipping Information – > Enter Payment Details -> Order Confirmation
There can be so many ways to build this journey:
Check out -> Enter Payment Details with check box for using earlier used Shipping Address -> Order Confirmation
I guess this explains what we are trying to do here.
Pricing
If there are different algorithms to determine the price on site and you want to see which algorithm fares better – then we run a A/B test to find the winner.
New Features
If there is a new feature going live – we test it to see how it resonates with the users. We build a hypothesis and that dictates the test. Example:
- Does auto play of video on product display lead to more conversions ?
- Does auto populating the payment information with a placeholder to enter CVV lead to more conversions?
- Does adding a visual indicator of shopping cart progress lead to more conversions ?
- Does adding new sections on pages (reviews, number of purchases etc.) lead to more conversions
- Will prompting for registration later on lead to more conversions?
So, in this way – we start with basic A/B testing and the move on to use A/B testing for tactical decision making. A/B testing is both art and science. Art is designing the test while implementing it is science.
Adobe Target , Optimizely & VWO helps in accomplishing few of these scenarios quite easily, but it requires a lot of commitment to implement such A/B tests for continuous decision making.
Note : There are some nuances of A/B testing that one should be aware of, these are often ignored. The below article captures these very well.