3 ms·
Seems like a straightforward solution would be to get people to opt-in by offering them credits, increased limits, early access to new features, etc. Universit
by jfarmer 7mo ago
Seems like a straightforward solution would be to get people to opt-in by offering them credits, increased limits, early access to new features, etc.
Universities have IRBs for good reasons.
- Aaargh20318 7mo agoA problem with this approach could be that you're now only testing the feature with the kind of people who would sign up for an A/B test. This group may not be representative of your whole user-base.
- jfarmer 7mo agoSo they'd need more robust experimental designs and statistical methods. They exist. And unlike the university context, there’s a glut of data. A basic technique: https://en.wikipedia.org/wiki/Inverse_probability_weighting https://en.wikipedia.org/wiki/Inverse_probability_weighting Or https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4384809 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4384809