4 ms·
> You don’t need all the status quo bias of null hypothesis testing. You don't have to make the status quo be the null hypothesis. If you make a change, you pr
by yorwba 1y ago
> You don’t need all the status quo bias of null hypothesis testing.
You don't have to make the status quo be the null hypothesis. If you make a change, you probably already think that your change is better or at least neutral, so make that the null. If you get a strong signal that your change is actually worse, rejecting the null, revert the change.
Not "only keep changes that are clearly good" but "don't keep changes that are clearly bad."
- scott_w 1y agoThis is a reasonable approach, particularly when you’re looking at moving towards a bigger redesign that might not pay off right away. I’ve seen it called “non-inferiority test,” if you’re curious.