This is a very good comment. Is this line of reasoning fleshed out and written up somewhere so I could point people to it? (Also, I would like to think more dee

This is a very good comment. Is this line of reasoning fleshed out and written up somewhere so I could point people to it? (Also, I would like to think more deeply about its implications) > What models/theories of customer behavior were your experiments designed to distinguish between? My guess is "none" because someone thinking about the problem scientifically would start with a single experiment whose results are maximally dispositive and go from there. This is how science is (at least, ought to be) done. This way, the goal is to always be improving your understanding of objective reality. > They wouldn't spend a bunch of time up-front designing 12 distinct experiments. [...] So it wasn't really an experiment in the scientific sense, but rather a kind of random optimization exercise: do 12 somewhat-less-than-random things and see which, if any, improve the metrics we care about. The problem is that a lot of AI salesmen tend to hype the "model-free" nature of "predictive" AI towards optimizing outcomes/goals, and people who don't know better get carried away with the bandwagon. Overly business-oriented people are susceptible to the ostrich mentality of not wanting to understand problems with bad tools -- they are too focused on the possibility of optimizing money-making. I find the movie "The big short" to be a fantastic illustration of this psychology. It's probably going to lead to a very bad hangover, but for the moment the party's still going on and nobody likes the punch bowl being yanked away.