Working with Rollout

Rollout is an Experience OS app that enables you to release new features gradually to reduce risk. Instead of pushing updates to all your users at once—expose a new feature to a small segment, measure business impact, and scale up safely. If anything goes wrong, do an instant rollback—no redeploy required.

How Rollout works

Your developer wraps the new feature code with a feature flag. Then, you create a new Rollout campaign to deliver this feature, and set the initial audience size. The Rollout campaign uses the Dynamic Yield Choose API call, sent with the Rollout selector, to determine whether or not your site visitors should get the new feature.

If everything goes as expected, you incrementally increase the number of recipients until 100% of visitors receive the new feature. However, at any point, you can easily roll back a release by pausing the campaign, and reactivate it when ready.

Managing Rollout

Install Rollout from the Experience OS app store. 

All your Rollout campaigns and templates are available within the app:  

• Features: Here you can manage all your new feature campaigns. Create new campaigns, edit existing ones, change campaign status, archive or delete campaigns, and more. You can search and filter your campaigns by status or label. Click the Columns dropdown to add more useful columns. Your last setting is always saved for future sessions.

• Templates: In this tab, you can manage any templates you've created for your feature rollouts.

Create a Rollout campaign

Step 1: Create the campaign

1. In Rollout, click Create Feature Rollout.
2. Enter a name, and optionally, add notes and labels.
3. Specify the API Selector. This selector is required in the API calls. Optionally, add an API selector group. 
   
   
   Tips for optimizing your Choose call performance:

   • Use selector groups. A selector group enables you to fetch the statuses of all relevant campaigns upon the session's first page view. We recommend that you cache all flag statuses for the remainder of the session if they're not intended to change during the session. This eliminates firing an API call on every page view.
   • We recommend using only 1 Rollout selector group per Choose call.
   • Don't mix campaign types. Include only Rollout campaigns in a single Choose request. 
4. Click Next to create a targeted experience.

Step 2: Create a targeted experience

1. Use the slider (or type in a number) to set the percentage of the target segment that should see the new feature. Note that some of this group won't see the Rollout feature if you allocate a portion of the segment to a control group. See step 5. 
2. Configure the targeting settings of the experience to determine when, where, and to whom the experience is displayed. When you're done, click Next. 
   Note the following:
   • Entering a rollout: Site visitors are first checked to determine whether they qualify for the campaign's experiences, in the order set in the campaign. As soon as a user qualifies for an experience, they are not checked for any additional experiences, whether or not they are included in the Rollout percentage.
   • Stickiness: Rollout campaigns are always Sticky for the user (multi-session). This means that if a user is included in the Rollout, they will always be served the same variation as long as the percentage slider remains the same or the percentage is increased. If the percentage is reduced or the campaign is rolled back, stickiness is reset and the same variation is not necessarily served. 
3. In the Variations tab, set the primary metric. Click Settings (gear icon) to configure the attribution window for the variation. 
   
4. Allocate a portion of your rollout delivery to a control variation to conduct a test and measure the feature's business impact. 
   
   This is checked by default, but you can deselect it for the following reasons:
   • If you don't want to conduct a test. That is, all the users in the segment get the feature variation.
   • When you are ready to fully deploy, you can deselect the checkbox and dedicate 100% of the defined rollout to your test variation. Don't deselect it during the Rollout, however, because every check/uncheck creates a new test version and resets stickiness, and the business impact data is affected.
5. You can add parameters to the API response. Click Add Parameters to use a Dynamic Yield template or one you created for this purpose. Note: We recommend that you avoid using variables that take data from the product feed. 
   
6. In the Status tab, set the experience status to Draft (default) or Active, and click Save Experience.
7. In Create Rollout, either save the campaign (publish if the experience is active) or create another experience (click the + sign, or duplicate the experience you just created). 
   

Edit a Rollout campaign

Click Edit (pencil icon) to edit any aspect of your campaign and experiences. If you are increasing or decreasing the rollout traffic allocation (that is, the number of site visitors targeted with the Rollout experience), note the following: 

• During the rollout: Changing the rollout traffic allocation doesn’t create a new reporting version, and therefore, doesn’t reset user stickiness. (However, if you decrease, some returning visitors won't enter the rollout even if they were in it before.)
• When you're ready for full deployment: In the Targeting tab, set the slider to 100%, and in the Variations tab, deselect the test (variation allocation) checkbox. A new reporting version is created with a single-variation report.
  Important: Do not deselect/reselect the allocation checkbox until the full deployment stage. Each time you do, a new test version is created, and user stickiness is reset. This affects the test measuring business impact.

Roll back and reactivate a Rollout campaign

If you encounter any issues with the deployment of your new feature, you can roll back the campaign until the issue is fixed, and then reactivate it. 

• To deactivate a campaign: In Rollout, click More (3 dots icon) in the campaign row, and then select Archive from the dropdown.
  
• To reactivate a campaign: In Rollout, click Unarchive (file box icon).
  

Reporting

Click View Report to access your campaign's Experience report. Rollout provides the following report types: 

• A/B tests: This is the report you see as long as the variation allocation checkbox is selected.  This report enables you to assess the results of a test between the test variation and control variation, and draw conclusions.
• Single Variation: This is the report you see when the allocation checkbox is not selected. This happens when you deselect the checkbox (say, upon full deployment, when only the Rollout variation remains, now at 100%) or if you didn't conduct a test to begin with. This report displays the usage attributed to the variation without testing it against an alternative.