3 ms·
I wonder if they even do their A/B testing right. In my experience (none at facebook) accidental p-hacking is rampant in the tech industry, with trials being
by darkpuma 8y ago
I wonder if they even do their A/B testing right. In my experience (none at facebook) accidental p-hacking is rampant in the tech industry, with trials being cut short or prolonged by the tester who's staring at a graph of the results in real time.
- michaelt 8y agoIf you have a change that improves your metric initially but damages it in the longer term, and your A/B test only detects the short initial effect, you can execute your A/B test perfectly and still get the wrong result.
- teleclimber 8y agoInterestingly that's exactly how algorithms make things go viral. They pick up on things that get a quick reaction with complete disregard for what happens in the long term. Jaron Lanier explains this in his talks [0]. So features on social media are decided based on short term gains and posts on social media are promoted like that too. It's like an entire industry forgot their parents warnings about thinking about the future. [0] https://www.youtube.com/watch?v=kc_Jq42Og7Q https://www.youtube.com/watch?v=kc_Jq42Og7Q