How does the A/B-Test Randomizer work - users to test versions allocation?
Hey, I have a few questions around this in regards to different scenario. I think this is crucial for the question, why we often cannot replicate winners, if we update a test
Scenario 1: Allocation in initial golive
#1 How are users allocated to test versions generally - via even/uneven or positive/negative DYIDs or else?
Scenario 2: Allocation after updating the test by changing only traffic allocation:
#2 Will only "new campaign participants" allocated to the test variations and nothing happens to the existing "campaign participants"?
Scenario 3: Allocation after updating the test by removing/adding test variations:
#3 Will only "new campaign participants" and "campaign participants from deleted versions" be (re-)allocated or all users?
Scenario 4: Allocation after updating the test by changing traffic AND removing/adding test variations:
#4 Basically the same question?
- Random. Not by DYID.
- It affects everyone, not just new users that are served with the experience.
- Same as #2
- Same as #2 and #3
Please sign in to leave a comment.