3 ms·

This answer is super spot on, and this will be the story of our lives for a while.

• TeMPOraL 6y ago
  It could be better than what we have now, assuming somewhat smart AI. Right now, data gives plausible deniability, and is the universal scapegoat - you get to do whatever you want, backing this up with misapplied A/B tests and telemetry, with zero statistical rigor. And if someone complains about your decisions, you point to these tests and that telemetry, and cry "we're a data driven company, and the data proves we're right".
  • xg15 6y ago
    I mean, it would help to be honest about the metrics that the A/B tests are evaluated against. If you optimize purely for growth and engagement, then your test results will reflect that - even moreso if your tests are scientifically sound. I'm continually surprised how companies manage to sell "let's try to keep users glued to the screen as long as possible" as doing something for the general good.
  • adrianN 6y ago
    I have little hope that the people not understanding their data today and lacking any statistical rigor will be able to train an AI to do better. Figuring out whether an AI gives you garbage results seems to be a harder problem than figuring out whether your own results are garbage.