Global Control Test

Global Control Test is a method for measuring the overall impact of your experimentation program. The measurement is done using an A/B test that splits your overall traffic into two groups (users who receive personalized Dynamic Yield experiences and those who don’t) and compares them.  

For example:

By comparing the performance of the Dynamic Yield Experiences and Control Group variations, we can estimate the total impact of experiences A, B, and C.

How does it work?

Setting up the Global Control Test

Step 1: Create the Global Control Test 

Create a Custom Code  campaign using the following settings:

1. Name the campaign Global Control Test.
2. Set the trigger as Page Load.
3. Set the frequency to Session.
4. Create an experience with the name Global Control Test.
5. Leave targeting conditions as they are, targeted to all users.
6. Create a variation called Dynamic Yield Experiences with the following JavaScript code:
   

   cleanGcgStorage();
   window.DYO.StorageUtils.set('_dy_cs_dygroup', 'Dynamic Yield Experiences', ['localStorage']);
    function cleanGcgStorage() {
     var GCG_STORAGE_NAME = '_dy_cs_gcg';
     var DYG_STORAGE_NAME = '_dy_cs_dygroup';
     var CG_VALUE = 'Control';
     var DY_VALUE = 'Dynamic Yield Experiences';
     var STORAGE_TYPE = 'localStorage';
     if (DYO.StorageUtils.get(GCG_STORAGE_NAME, [STORAGE_TYPE]) === CG_VALUE) {
       DYO.StorageUtils.remove(GCG_STORAGE_NAME, [STORAGE_TYPE]);
     }
     if (DYO.StorageUtils.get(DYG_STORAGE_NAME, [STORAGE_TYPE]) === DY_VALUE) {
       DYO.StorageUtils.remove(DYG_STORAGE_NAME, [STORAGE_TYPE]);
     }
   }

7. Create another variation called Control Group with the following JavaScript code:
   

   cleanGcgStorage();
   window.DYO.StorageUtils.set('_dy_cs_gcg', 'Control', ['localStorage']);
   function cleanGcgStorage() {
     var GCG_STORAGE_NAME = '_dy_cs_gcg';
     var DYG_STORAGE_NAME = '_dy_cs_dygroup';
     var CG_VALUE = 'Control';
     var DY_VALUE = 'Dynamic Yield Experiences';
     var STORAGE_TYPE = 'localStorage';
     if (DYO.StorageUtils.get(GCG_STORAGE_NAME, [STORAGE_TYPE]) === CG_VALUE) {
       DYO.StorageUtils.remove(GCG_STORAGE_NAME, [STORAGE_TYPE]);
     }
     if (DYO.StorageUtils.get(DYG_STORAGE_NAME, [STORAGE_TYPE]) === DY_VALUE) {
       DYO.StorageUtils.remove(DYG_STORAGE_NAME, [STORAGE_TYPE]);
     }
   }

8. Set the allocation to 95% for Dynamic Yield Experiences and 5% for Control Group.
9. Ensure Variation Stickiness is set to Sticky for the User (multi-session).
10. Publish the campaign.

Step 2: Create the evaluator

The evaluator is used to target experiences to the Dynamic Yield Experience group or the Control Group.

1. Create an Evaluator with the name Global Control Test Group.
2. Set the value type to String.
3. Set Expected Values to Dynamic Yield Experiences and Control Group.
4. Insert the following code:
   

    (function() {
     window.DYCS = window.DYCS || {};
     var deferred = DYO.Q.defer();
   
     if (DYCS.inGlobalControl) return DYCS.inGlobalControl;
   
     getGCCookie();
     return deferred.promise;
   
     function getGCCookie () {
       var GCValue = DYO.StorageUtils.get('_dy_cs_dygroup', ['localStorage']) || DYO.StorageUtils.get('_dy_cs_gcg', ['localStorage']);
       if (GCValue) {
         DYCS.inGlobalControl = GCValue;
         deferred.resolve(GCValue);
       } else {
         setTimeout(function() {
           getGCCookie();
         }, 10);
       }
     }
   })();

5. Save the evaluator.

Step 3: Add experiences to the Global Control Test

Adding experiences to the Global Test is an ongoing process. If you have a personalized experience, target it (using the evaluator you just created) to the Dynamic Yield Experiences group. This ensures that only users who are part of the 95% population receive this experience. 

If the campaign required a default experience (served to all traffic, including the 5% control group), create the experience and serve it to the Control Group using the same evaluator. 

Best practices: What should be measured?

• Recommendations, personalized experiences, and experiences running dynamic allocations should be served to the Dynamic Yield Experiences group.
• A/B tests, during the testing period, should not be served as part of the Global Test, as this is an exploration mode. However, after a winner is declared, you should pause all losing variations and serve the winning variation to the Dynamic Yield Experiences group.
• If you must serve content to the Global Control group (for example, if you have a personalized hero banner, but can’t afford to serve an empty slot) create an experience with a single variation as a fallback.
• Experiences that do not modify the user experience (fire an event or implement a cookie) should not be included in the Global Test. 
• Experiences that must be served and are not optimized (say, cookie consent notification) should not be included in the Global Test.