Outliers detection – ridding the extreme events threatening your A/B tests

What would happen if one day, a huge deviation in visitor behavior were to contribute to a variation’s success, like a random user making a one-of-a-kind purchase?

This type of activity might lead to a false score, leveraging data that does not reflect normal site behavior to predict the future outcomes of an experiment. Therefore, those tasked with optimizing experiences must be sure to account for and properly handle these extreme events within their campaigns. As not doing so will negatively skew results and tip the scales in favor of variations that may not, in fact, shape up to be the best in the end.

Learn about outliers, how to detect them, and how to filter them out in this blog post.