Predictive Targeting

When you run an A/B test, the Dynamic Yield predictive analytics engine runs in the background, searching for hidden personalization opportunities.

The predictive analytics engine solves one of the biggest problems of A/B testing: Tests optimized for the average user. In A/B tests, a winning variation is selected based on the preferences of most users. This means that if most users significantly prefer variation A, when the test concludes, variation A is served to all users. However, often there are subsets of users with specific preferences that are not the winning variation. So you can improve your results if you serve a different variation to that set of users who prefer a less popular option. 

Predictive targeting ensures that even tests that conclude without an overall winner might have a chance to generate uplift by serving a “losing” variation to an audience that actually prefers it.

The following image shows a test that resulted in an overall uplift of -51.7%. Predictive targeting suggests a new course of action: If variation A is served to the Referral Traffic audience while all other users receive the control group, the expected uplift is +11.4%.

How it works

1. Predictive Analytics runs in the background of every A/B test you run, scanning all audiences, and surfacing those who prefer a “losing” variation. Only courses of action with enough significance are evaluated and prioritized in terms of significance and expected uplift.
2. When an opportunity is identified, you are notified in the report and by email. The table displays the targeting groups, the variations that should be served to each, and the uplift the course of action is expected to generate.
3. You can accept the personalization opportunity by clicking APPLY if you want to take advantage of this opportunity. Then, one of the following occurs:
   • If the Control Group is one of the variations:
     A new targeting condition is added, limiting the serving of the experience, so those who preferred the control group are not served with this experience.
   • If the Control Group is not one of the variations:
     The experience is duplicated, and a new targeting condition is added to the new experience. All variations but one are paused in one experience, and all variations but (a different) one are paused in the other experience, so one targeting group is served with one variation and the rest with the other.
4. Click the Additional Options menu (3 dots) to see the following options:
   • Review before publishing: Redirects you to the form with the setup that clicking on APPLY would create. This enables you to double-check the setup and make any changes before publishing.
   • Breakdown report: Enables you to see how the relevant audiences performed. You will see that there’s a different leader when filtering the report.
5. Click More Options under the table to display any additional courses of action and the common A/B testing course of action (serving the leading variation to all users).

Statistical significance of personalization opportunities

Personalization opportunities require high significance to ensure valid suggestions. The suggested course of action must be better than serving the leading variation to all users by at least 1%, with a confidence of at least 95%.