Dynamic allocation is one of the traffic allocation modes that determine how to allocate variations of experiences across your traffic. The Dynamic Allocation mode automatically determines what percentage of traffic each variation will serve. It leverages a Multi-Armed Bandit strategy, which means that variations that performed better in the past, are allocated more traffic. The best variation is gradually served to more and more of your visitors as data is collected. Over time, the leading variation might be replaced if a different variation begins performing better.
Unlike A/B tests, dynamic allocation aims to maximize the performance of an experience as quickly as possible, with less focus on reaching statistical significance and more focus on optimization and speed.
This allocation mode is ideal for promotions that are changed frequently, as it quickly shifts traffic to better-performing variations automatically.
Dynamic Allocation algorithm
Variations are initially allocated equal traffic. As users interact with the variations, we use the primary metric to calculate the Probability to be Best score and, over time, increase exposure of the more successful variations. The more time has elapsed, the more heavily the score is weighted.
At least once every hour, Dynamic Yield recalculates the scores and assigns allocations (globally for all experiences). This ensures that optimization is done quickly, to make the most of the limited lifespan of creatives.
Measuring Dynamic Allocation overall experiences performance
For experiences using A/B test allocation, the Probability to be Best score typically reaches statistical significance and declares one “winning” variation. However, in experiences using Dynamic Allocation, as they are designed to change the allocation of the variations constantly, we would like to measure the overall performance of the experience, regardless of the variation served. For example, if you have a hero banner with multiple variations, optimizing for CTR, you would like to see the overall performance of the banner element, aggregating impressions and clicks for all the variations chosen by Dynamic Yield to populate the banner.
We recommend allocating some traffic to a control group that selects a random variation. This enables you to compare sessions served with a control group and the ones served with Dynamic Allocation, measuring the impact of the automatic allocation that Dynamic Yield manages vs. the static control group in the Experience Reports.
As long as the control group is assigned a random variation, and is assigned 10% of your traffic or less, the traffic is taken from the “hidden” control group (see more under Explore vs. Exploit). Therefore, we recommend allocating 10% of your traffic to such a control group, as it does not affect your experience negatively in any way and has the benefit of allowing you to measure your campaign.
For details about how to define a control group, see Using the Control Group.
Effects of editing experiences
As dynamic allocation is often used for frequently edited campaigns, we do not reset the algorithm every time a change is made. If a variation is edited, added, or deleted, we give it a relative Probability to Be Best score according to the number of variations (for example, 25% if 4, 33% if 3). The existing variations are allocated the remaining traffic according to their previous performance.
For example, let’s say you have three variations: A, B, and C. A is performing best, followed by B and then C.
- If you delete C (the worst variation), A will still get more exposure than variation B.
- If you edit variation C, A will still get more exposure than B, but C will be served to 33% of traffic in the first hours, allowing the system to evaluate the new version of variation C.
- If you add a new variation (D), it will initially be allocated 25% of the traffic, allowing the system to evaluate it, while still allocating A more than B, and B more than C.
In short, when using dynamic allocation, you can and should edit your variations to weed out losing variations and try new ones. The algorithm is designed to constantly adjust and optimize the allocation so you get the best possible results.
Note: The Probability to Be Best is always calculated from the last edit of a variation until the current time, regardless of the time selected in an experience report. This might cause seeming discrepancies between the Probability to Be Best and the primary metric when viewing reports in different time ranges, especially after editing an experience.
Explore vs. exploit
Explore and exploit are terms that define how much is "wasted" on learning, and how much on taking advantage of what is learned. In dynamic allocation, the explore and exploit ratio is 10/90. Meaning, 10% of traffic will be served a random variation ("hidden" control group), and 90% will be served the better-performing variation. This enables Dynamic Yield to direct most traffic to the leading variation, but still learn about "losing" variations and enable losing variations to bounce back.
When using a control group that serves a random variation, Dynamic Yield reduces the explore rate, as it can use the control group for this purpose.