3 ms·
Are you really sure you're not just fooling yourselves with your randomized controlled trials? As Feynman famously said, the easiest person to fool is yourself.
by semi-extrinsic 4y ago
Are you really sure you're not just fooling yourselves with your randomized controlled trials? As Feynman famously said, the easiest person to fool is yourself. And in business even more than science, you might even like the results.
Have you ever put this data up against something similar to the peer review system in academia, where several experts from a competing deparment (or ideally competing company) try to pick your results apart, disprove your hypothesis?
- johnthewise 4y agowell, certainly it's possible to fool yourselves with A/B testing, it doesn't mean you must be fooling yourselves. I've also seen similar results in recommendation settings in mobile gaming, not once but over and over again across portfolio of dozens of games/hundreds millions of players. You don't need to predict 20% better on whatever you are predicting to get a 20% increase in LTV and it's even better if you are doing RL since you are optimizing directly for your KPIs
- Kiro 4y agoWhy are you doubting them? Nothing in the comments indicates what you are suggesting.
- semi-extrinsic 4y agoNullius in verba. That is how good science has always been done.