An A/B test serves two purposes:
- It allows us to scientifically measure the impact of the Avatria Convert recommendations on site conversion events in comparison to the default sort order
- It provides business users a very simple method for enabling the Avatria Convert ranking recommendations on the product listing pages of their choosing, without requiring additional development
Since we want to measure eCommerce related conversion metrics (e.g. carts, checkouts, purchases), we recommend using Google Optimize for implementing the A/B tests. Google Optimize is part of the Google Analytics Suite, and allows for comparing your experiment variations using metrics & events already captured in your existing Google Analytics tagging implemention. Additionally, Google Optimize can quickly be integrated into your site, without development assistance, using Google Tag Manager.
Setup Google Optimize
If you have not already enabled Google Optimize on your account property, the instructions below can quickly get you setup using Google Tag Manager:
https://support.google.com/tagmanager/answer/7164339?hl=en
Create Experiment Variants
The A/B Test experiment for the POC should consist of two variants:
- Original: The site in normal state with all category locations sorted using the default sort order
- Variant 1: Same as original, except a few select category links located in the site header & footer will be updated to include the appropriate sort query parameter value to enable the Avatria Convert recommended sort order
The updates required for Variant 1 can be made easily by using the Google Optimize Visual Editor.
For each category link in the test, simply use the WYSIWG editor to select the link and edit the ‘href’ attribute value to include the proper query parameter value to enable the Avatria Convert ranking order.
Note: Remember to encode the URL query parameters, when updating the link values.
Configure Experiment Objectives
At a minimum, the experiment should have the ‘Transactions’ objective associated with it. This objective should be available once you link your Google Optimize Container with your main Google Analytics View.
If you don’t see the ‘Transactions’ experiment listed after linking with GA, ensure your Google Analytics Property has ‘Shopping’ selected as it’s Industry Category within the ‘Property Settings’ section.
If the proper GA tagging is in place, it is recommended that you also add additional objectives for ‘Cart’ and ‘Checkout’ events. Please contact the Avatria Convert team if you have questions regarding how to set this up.
You may also add other objectives to the experiment, as you see fit.
Configure Experiment Targeting
Before enabling the experiment, you must also configure the targeting rules. Given we only have two variants (Original & Variant 1), it is recommended to target 100% of visitors. By default, the weights of visitors shown each variant will be 50/50. You can adjust this if you’d like, however, we don’t recommend showing the Avatria Convert variant (Variant 1), to more than 50% of visitors.
The experiment evaluation configuration (shown under When) should be set to ‘Evaluate on page load’.
After everything is setup, you should click ‘Save’ and use the ‘Start Experiment’ to enable your test.