The Experience Report report displays the results of your experiences. Results are broken down by variation, but you can also see results for the entire experience for Dynamic Allocation tests. You can view the impact the experience had on primary and secondary metrics, and segment the results by audience.
The structure of the report changes depending on whether the experience is an A/B test, a single variation without a control group, or uses Dynamic Allocation.
A/B test results display the timeframe of the Current Version by default. This is the data that was collected since the test version started, until last midnight. The data is updated daily at 9 am in the timezone of the site. This is the most accurate way to look at your A/B test results. If there are any previous versions of the test, you can select them from the dropdown to change the view. Alternatively, you can view results for custom time frames, with convenient presets such as Last 7 days or Yesterday. For user-based tests, custom time frames will estimate the number of Unique Users during the timeframe, so we recommend looking at the entire test version before acting on the test results.
This section appears only if the experience is using Dynamic Allocation. It shows the overall performance of the experience, regardless of the variation that was served. For example, if you have a hero banner that is serving multiple variations, optimizing for CTR, this view will show you the overall performance of the banner element, aggregating impressions and clicks for all the variations which were chosen by Dynamic Yield to populate the banner.
If you use a control group, it also offers a comparison between the sessions that were served with a control group and the ones served with Dynamic Allocation, measuring the impact of the automatic allocation that is managed by Dynamic Yield, vs. the static control group.
The graph shows over-time results for your primary metric, and a set of fixed metrics (purchases, impressions, users, AOV,). You can view metrics in one of 2 modes:
- Daily results: the performance on each individual day.
- Cumulative results (only in A/B tests): each point in the graph shows you the data from the beginning of the test version until that day.