Multivariate Testing with Sitecore

In this post I will explain how you can set up multivariate testing with Sitecore, if you don’t know about what I am talking, checkout the link below will give you good idea about A/B and Multivariate Testing

https://www.optimizely.com/optimization-glossary/multivariate-test-vs-ab-test/

The Scenario

We have website to sell products, and when the user needs inquiry about specific product, there is a contact us page, it has a form where the user can fill out some fields then he will submit the form, lately we found out the number of form submissions is not as we expected, we did some investigation we found out the users are reaching the contact us page but because the form has some useless/extra fields, so the users are not submitting the form, so we created different variations for the form by removing and adding some fields, then using out of the box feature of multivariate testing in Sitecore we will check which one will give the best experience for the user.

Step 1: Define 4 variations for the form and added to the original form

Original Form:

  1. First Name
  2. Last Name
  3. Email
  4. Mobile
  5. Country

Form A:

  1. First Name
  2. Last Name
  3. Email
  4. Mobile

Form B:

  1. First Name
  2. Last Name
  3. Email
  4. Mobile
  5. Budget

Form C

  1. First Name
  2. Last Name
  3. Email
  4. Mobile
  5. Property Type

Form D

  1. First Name
  2. Last Name
  3. Email
  4. Mobile
  5. Nationality

Step 2: Define 5 Goals, one for each form in the marketing control panel

Goals.png

Step 3: Register goal, and we have done that by code on form submission, we created function to trigger the goal and passing the goal id

public bool TriggerGoal(ID goalId)
 {
 bool triggeredGoal = false;
 try
 {
 if (!Tracker.IsActive)
 {
 Tracker.StartTracking();
 }

if (Sitecore.Analytics.Tracker.IsActive && Sitecore.Analytics.Tracker.Current.CurrentPage != null)
 {
 var goalItem = Sitecore.Context.Database.GetItem(goalId);
 if (goalItem != null)
 {
 var goal = new PageEventItem(goalItem);
 var pageEventsRow = Tracker.Current.CurrentPage.Register(goal);
 pageEventsRow.Data = goalItem["Name"];
 Tracker.Current.Interaction.AcceptModifications();
 triggeredGoal = true;
 }
 }
 }
 catch (Exception ex)
 {
 Sitecore.Diagnostics.Log.Error("Failed to trigger goal: " + ex.Message, ex);
 }
 return triggeredGoal;
 }

Step 4: Adding the variations for the components from Sitecore

From Content tree -> go to Contact us item -> Presentation -> Details -> Click on edit -> Controls – > Choose the contact us component -> Click on Test

and there you can add your variations, check image blow :

Presntation2.png

Note:I have different structure in Sitecore, so i am using data-source 
and the form fields are configurable, i am using a multi-list fields to 
show different fields  for each form, but you can do variations by 
components by check the enable variation of component design.

once you have done that, you will see the contact us form has 5 variations as below:

Presntation1.png

Step 5: Start the Test

on contact us item you will see that new components has been added to the item, so you need to start the test by clicking “create test” then publish the site.

CreateTest2.png

when you click on create test, it will show you preview for the test, go to variables tab there are some settings where it shows that you have contact us form component with 5 variations, and the percentage of visitors exposed to test, in my case i put it 100% that means 20% of the visitors will see each form (form A 20%, form B 20% …etc), you can change this percentage if you want :

test1.png

Note: if you put the percentage for example 90% that means 90% of the 
users will see the original form and 2.5% of the users will see the other 
forms.

in objective tab you can choose what is your test objective and for how long you need to run the test, if it is an A/B Testing form test objective usually will be the goal from drop down you can select the goal that you want to achieve, but in my case i am using multivariate testing so and i have 5 goals so i will keep it as is based on user visit “Trailing Value/Visit“, i will keep the testing for 7 days as well, also i change the setting for the way of pickup the winner i want to select the winner manually as below, after that you click start test then you publish the website.

Test33.png

Step 6: It is time to collect the data 🙂

This is out of the box feature in Sitecore, so automatically it will start showing different variations of forms for the users based on the percentage that we have defined, when you go to experience analytics from launchpad,and you will end-up with the following result where it shows you the results that you want to see, and any time you wish you can stop the testing by going to Experience Editor then navigate to contact us page.

2018-07-01 22_11_35-mafpjumpsrv3.MAFGROUP.LOCAL - Remote Desktop Connection.png

Conclusion

In the end i hope that i covered all the points related to multivariate testing, and I hope you find this post helpful, If you have any additional knowledge on this subject, comments or questions, please let me know in the comments section below.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s