4 ms·

Can this be solved by setting p=0.50? Make your expectations explicit instead of implicit. 0.05 is completely arbitrary. If you are comfortable with a 50/50 ch

by jjmarr 1y ago

Can this be solved by setting p=0.50? Make your expectations explicit instead of implicit. 0.05 is completely arbitrary. If you are comfortable with a 50/50 chance of being right, make your threshold less rigorous.

scott_w 1y ago
I think at that point you may as well skip the test and just make the change you clearly want to make!
- bigfudge 1y ago
  Or collect some data and see if the net effect is positive. It’s possibly worth collecting some data though to rule out negative effects?
  - scott_w 1y ago
    Absolutely, you can still analyse the outcomes and try to draw conclusions. This is true even for A/B testing.