> These are vastly different probabilities When it comes to certainty about an outcome (i.e. a hypothesis), they're not really that different. Probabilities ar

> These are vastly different probabilities When it comes to certainty about an outcome (i.e. a hypothesis), they're not really that different. Probabilities are not linear, and start at 0.5. In log-odds terms were talking about 2.19 vs -1.04... not really a shocking amount of information. Sure in terms of gambling odds this might be useful but not for making any concrete statements about the world. To your point: > I bet most companies would die to turn up conversion rates up even 1 or 2 percent. This is talking about two very different things. I would also be happy if I had a known 2 percent increasing in my stock performance. We're talking about quantifying uncertainty (though these criticism hold even in a frequentist frame work). I certainly wouldn't "die for" 2% absolute increase in my confidence about something (unless I'm 98% confident already!) We're talking about our confidence in a hypothesis. At no company I've worked at would I be anywhere near comfortable stating that a 0.9 probability that A > B is enough to call an A/B test on. I'm going to be very skeptical of any analysis I'm doing if mode coefficients have anywhere near that probability of a sign error. If you have no other information, then sure if I am 90% confident that one choice is stronger than other I'll go with the limited information that I have. However anyone with a statistics background would view either of these numbers as being very uncertain. That's arguably why these numbers are so good for betting, you want there to be a lot of uncertainty when gambling or it's no fun. But as far as statistics is considered both 0.9 and 0.26 are essentially uninteresting numbers, that lead to very low confidence in the conclusion being drawn.