3 ms·

Frequentist methods are not dependent on the researcher's state of mind, but on what actions he would have taken if things turned out differently. I've got sli

Frequentist methods are not dependent on the researcher's state of mind, but on what actions he would have taken if things turned out differently. I've got slides that illustrate exactly your example. Look at the graph below: https://www.chrisstucchio.com/pubs/slides/gilt_bayesian_ab_2015/slides.html#18 https://www.chrisstucchio.com/pubs/slides/gilt_bayesian_ab_2... Your different conditions simply represent the probability of the blue line crossing two different other lines. Frequentism isn't stupid, but you do need to be really smart to use them correctly. At VWO we switched to Bayesian because our customers are marketers, not statisticians.

loup-vaillant 10y ago
> Frequentist methods are not dependent on the researcher's state of mind, but on what actions he would have taken if things turned out differently. That's exactly what I had in mind. It amounts to the same, really. Granted, the researcher's decision process is bloody important: it determines what the researcher will be doing. But once you know what has been done, that decision process has no further influence over the experimental results —only the actual experiment does. Of course, this is all knowing what priors we had in the first place. Since one's priors tend to influence one's decision process, one may chose to ignore one's own priors, and deduce them back from one's decision process. In that narrow sense, different decision processes do yield different conclusions. Going this route amounts to ignore relevant information however, and that would be stupid. > Frequentism isn't stupid, but you do need to be really smart to use them correctly. I believe the "you have to be smart" argument have also been used to attack Bayesian methods on practical grounds. It's a valid attack either way, but a bit weak for my taste. When you apply probability theory, you have to make a logical error to draw the wrong conclusion. The only problem left is prior beliefs, and I don't believe we can escape the need for them. I'm not sure mucking Frequentist methods up requires a logical error. If it indeed doesn't, then we can safely declare Frequentism "unsound" and move on, don't you think? > [VWO slides] Hmm, so Bayesian methods are easier to explain… interesting, thanks for the link.

loup-vaillant 10y ago

> Frequentist methods are not dependent on the researcher's state of mind, but on what actions he would have taken if things turned out differently. That's exactly what I had in mind. It amounts to the same, really. Granted, the researcher's decision process is bloody important: it determines what the researcher will be doing. But once you know what has been done, that decision process has no further influence over the experimental results —only the actual experiment does. Of course, this is all knowing what priors we had in the first place. Since one's priors tend to influence one's decision process, one may chose to ignore one's own priors, and deduce them back from one's decision process. In that narrow sense, different decision processes do yield different conclusions. Going this route amounts to ignore relevant information however, and that would be stupid. > Frequentism isn't stupid, but you do need to be really smart to use them correctly. I believe the "you have to be smart" argument have also been used to attack Bayesian methods on practical grounds. It's a valid attack either way, but a bit weak for my taste. When you apply probability theory, you have to make a logical error to draw the wrong conclusion. The only problem left is prior beliefs, and I don't believe we can escape the need for them. I'm not sure mucking Frequentist methods up requires a logical error. If it indeed doesn't, then we can safely declare Frequentism "unsound" and move on, don't you think? > [VWO slides] Hmm, so Bayesian methods are easier to explain… interesting, thanks for the link.