What is A/B Test traffic allocation
Also known as manual allocation, the percentage of traffic allocated to each variation is defined before making the campaign live and remains static until it ends. If at any point changes are made (for example, adding additional variations or adjusting the allocation percentages), a new version is created and the campaign is restarted. Once the Minimum Test Duration and Significance Level have been reached, a winning variation (or lack thereof) can be declared.
What is Dynamic Allocation
The percentage of traffic allocated to each variation is decided by Dynamic Yield as the campaign progresses. It leverages a Multi-Arm Bandit strategy, which means that variations that are performing better will be allocated more traffic. The better variation will gradually be served more and more traffic as additional data is collected. The better variation can change over time with other variations taking the lead and drawing more traffic.
When to use which method
◊ Are you looking to make long term changes to your site?
This is the classic indicator that an A/B Test should be used. Statistically significant data should be accumulated before making any big or strategic decisions as to how to proceed with this change.
◊ Are you running a short-lived campaign, like a holiday banner?
Here, the objective is to increase the KPI, by engaging as many users as possible with the better performing variation - therefore Dynamic Allocation is recommended. This ensures that the variation driving the highest results will be served more frequently, increasing results at a much quicker rate.
Defining the Allocation Method
Please sign in to leave a comment.