To ensure clean and reliable test results, each test is managed in versions. This enables you to make changes to a running test without mixing the pre- and post-change data in a way that affects the result. When a new test version is created, both the allocation and the Probability to Be Best result are reset. The report lets you switch between the current version (default view) or any previous test version that ran in this experience.
Note: Previously, only experiments with the A/B test allocation method used to be version managed. The new generation experience report, released in early 2024, applies the same behavior to all experience report types.
Learn more about selecting report version in Experience Reports.
Creating a new test version
The first test version is created when you launch an A/B test experience for the first time. If you edit the experience, you are notified that a new test version will be created, and you can select whether or not select to proceed with the new version.
We recommend creating a new test whenever the changes are expected to affect the test results (for example, changing the image or copy). For smaller changes that probably won’t affect the results, you can continue using the same test version. For example:
- Changing the tracking URL parameter of a target URL.
- Changing the trigger from time on page of 15 seconds to time on page of 14 seconds.
- Changing the backdrop color of an Overlay.
In the following cases, a new test version is always created:
- The main metric changed.
- At least one variation was removed, added, paused, or unpaused.
- Allocation of variations changed.
- Stickiness was changed.
- The status of an experience changed (say, from active to draft).
- The campaign is archived.