Creating an A/B test for a site becomes so easy using Adobe Target. However, one case easily overlook certain aspects of the test. This can lead to a bad experience for the site visitors. This post aims to share few tips to ensure a smooth implementation end to end.

Strategize the test

This means at a minimum – think and document the below aspects of the test :

• Objective of the test : Most of the times this will be an improvement on metrics like Clicks, PageViews etc.
• Site Element you want to test : You may not want to test a element which is dynamic and will be taken off from the site during the next content update
• Finalizing the duration of the test : You can use the Adobe Target sample size calculator to get an idea of this. This is to ensure mistakes like -Declaring a winner before the test is complete or continuing a test endlessly are avoided.
• Finalizing the Audience for the test: It is common for companies to do a test on 20% of population and have the rest of the population experience the winner on a big day/promotion

Implementation

While creating a test is very easy. One should be careful on the below aspects:

• Check for flickers which is control version getting displayed before the experience B. I will write a separate post on how to solve the flicker issue on Target
• Check whether all pages ensure the same experience. This is very much needed when you are performing a test on items which can be seen across pages like headers and footers. All you have to do is go to Configure – > Page Delivery and add the domain/Url/ Path
• Check the site UI elements : The UI elements can easily fall out of layout or grid. So you have to ensure things like grids or image containers are resized. This ensure a good experience for the visitors

Reporting & using the results:

• Ensure that the test is deactivated once the duration is passed. Check the lift and the confidence. A lift of >10% with a confidence of 90% is good to implement (opinion). This actually depends on the organization as to what they want the confidence level to be . Some can even peg it 95 or 98% . Implement what you have learned from the A/B test and optimize the site.

Let me know your thoughts as well