MVT using Publishing Targets

I have been involved in a few enterprise projects where unfortunately I cannot easily enable XDB architecture and fully leverage Sitecore capabilities for MVT testing, therefore, it comes to the idea of getting creative with what I got and find an alternative way of helping the marketing team testing the effectiveness of their new content and the new functionality that they have in mind….

I guess that it is pointless to mention, the advantages of the Testing culture and the idea behind fail quickly… however all these principles & features are covered with Sitecore MVT and in case you have XDB enabled in most of the cases I would strongly suggest you to use the built-in features…

In the specific project circumstances, I have been asked to test the effectiveness of some new “label” within the Booking funnel to see the impact on conversion….

For the Tracking, we were not using XDB and Sitecore analytics but Google Analitycs do a good job tracking Conversion and events…

For trying the different variants, I have split the sitecore change across the 4 Publishing Targets that I have for scalability reasons…  you can read more about publishing target here

My architecture requires a CDN / Proxy (Akamai in my specific case) to split the traffic across my 4 Content Delivery servers and have a publishing target different for each of the Content Delivery server…

Each Variant can be assigned to one or multiple CDs so that you can have a simple A/B testing or a more complex MVT….

With this architecture and this setup, I was able to allow the marketing team to test their content experiment and help them to find the best variation and the winning experiment even without having to configure Sitecore analytics, XDB and so on, but just leveraging the existing architecture….

The change that you would need to implement is to send via a custom dimension to GA the name of the Content Delivery server for each page view / event / transaction…

the code to track the CD, would be as simple as:

ga('set', 'cd1', 'Level 1');

obviously, this approach comes with some trade-off….  since Marketing users cannot really know which test is running and risk to override a running test doing a simple content publishing…. Also from the reporting point of view, you would need to keep track of the publishing that your content team is doing to be sure that anything else is affecting your experiment…. Also, need to remember to switch off your experiments and align all the content on the publishing targets to have a consistent web site….

 

Untitled Diagram

Advertisements

Sitecore MVT for Beginners

MVT stands for multivariate testing if you are not familiar with the concept and you know A/B testing is more or less the same thing and in the case you have never heard about it, the principle behind it is to expose your web site visitors to multiple variations of the website content/functionalities to TEST  what is the variant that perform better.

It is important to specify that Sitecore generate the TESTS variation server side and it requires XDB enabled to track analytics and to help you to understand the more effective variant.

There are two way that Sitecore use to deliver MVT functionalities:

Component Based Testing

you can configure it either in Page Editor Mode either from the presentation details here you can specify all the variations that you want to expose your customers and it is very easy to configure as you can see in the following screenshots.

testcomp

mvtcomponents

Page Based Testing

From here you can control all the running Tests and you can see Sitecore suggestions for the Pages to test…

When you create a new Test you will be asked for the page to Test and the Variations of the page to test (you can either use different pages or create different versions of the same page)

You can than define:

  1. The Percentage of the traffic visiting the page to run the test
  2. The Success criteria of your test (eg. if the user is visiting more pages / increasing the engagement value or if the user is triggering a specific Goal/Event eg. conversion rate
  3. The Duration of your test (for how long you want to run the test)
  4. The confidence level, when do you want Sitecore to decide the winner of the test and serve automatically the winner of the test

Happy Testing!

ExperienceOptimisation.png

checkouttest

testdefinition

testwinner