4 ms·
I have a simpler reason for the same belief. You can fake everything except a well designed A/B test. At FAANG scale, a statistically significant A/B test requ
by screye 3y ago
I have a simpler reason for the same belief.
You can fake everything except a well designed A/B test. At FAANG scale, a statistically significant A/B test requirement will stop the worst fraud before it hits the user.
- onlyrealcuzzo 3y agoAnd also - you can somewhat take care of bugs by evenly distributing them to your test & control group.
- fshbbdssbbgdd 3y agoFAANG engineers: “hold my beer” Seriously though, as a person who has built related systems at FAANG, yes this problem exists there. Your beautiful cathedral of an A/B testing framework is covered in knobs that are just perfect for p-hacking.