You’re making an interesting point here. I’m resistant to it because some analysis I do (say, inferring atmospheric composition from the spectra of light you s

You’re making an interesting point here. I’m resistant to it because some analysis I do (say, inferring atmospheric composition from the spectra of light you shine through the atmosphere) does fit rather well in the Bayesian setup. In a very real sense, if you are doing a bunch of these inversions and not using Bayesian methods, you are leaving RMSE on the table. Here, the “features” are the spectra you sense and you add some basic prior information based on historical expectations. You don’t have that much wiggle room to invent new variables to confuse the analysis. But the situation changes if you have woolier problems with human judgement in finding “features” — the variables in the prior or likelihood in the spam example you give. And you can fool yourself badly! E.g., to go with your market example, I have two models for the stock market, and 1 month of historical data strongly favors model 1 over model 2, so I bet heavily on model 1 as if that’s the only possibility. But it wasn’t, and I lose. The area of causal inference uses a slightly different set of tools that gets outside the Bayesian setup, including the Do() operator, which allows for manual intervention to probe the linkages in systems like these. There’s a lot of activity there as we’re looking for systems that learn reliably. I don’t know if you listen to podcasts, but Judea Pearl was recently on Sean Carroll’s (excellent) Mindscape podcast. They discuss this at some length, it was very nice. https://www.preposterousuniverse.com/podcast/2022/05/09/196-judea-pearl-on-cause-and-effect/ https://www.preposterousuniverse.com/podcast/2022/05/09/196-... PS: it’s enjoyable to see your comments here.