How does the A/B-Test Randomizer work - users to test versions allocation?

Follow

Kamal Abdul karim

• March 17, 2020 11:31
• Edited

Hey, I have a few questions around this in regards to different scenario. I think this is crucial for the question, why we often cannot replicate winners, if we update a test

Scenario 1: Allocation in initial golive

#1 How are users allocated to test versions generally - via even/uneven or positive/negative DYIDs or else? 

Scenario 2: Allocation after updating the test by changing only traffic allocation:

#2 Will only "new campaign participants" allocated to the test variations and nothing happens to the existing "campaign participants"? 

Scenario 3: Allocation after updating the test by removing/adding test variations:

#3 Will only "new campaign participants" and "campaign participants from deleted versions" be (re-)allocated or all users?

Scenario 4: Allocation after updating the test by changing traffic AND removing/adding test variations:

#4 Basically the same question? 

Thanks

0

• Official comment

  

  Gidi Vigo

  • March 17, 2020 13:04

  1. Random. Not by DYID.
  2. It affects everyone, not just new users that are served with the experience.
  3. Same as #2
  4. Same as #2 and #3 

  Comment actions Permalink

Please sign in to leave a comment.