4 ms·

I'd say classical statistics lets you deduce, based on some knowledge that something is the case, the probability of an event. It doesn't let you induce the pro

by bbctol 10y ago

I'd say classical statistics lets you deduce, based on some knowledge that something is the case, the probability of an event. It doesn't let you induce the probability of something being the case (that's why we have all this indirect dodging with p-values.) So I think there is a fundamental difference between "What are the odds of rolling a six" as "What are the odds of that event happening," and "What are the odds the Sun revolves around the Earth" as a question about whether or not something is true. That's what the author seems to mean by "either it does or it doesn't," though it isn't worded that well. Even if the die has already been rolled, we know that it could have been a 4 or a 5, and have good knowledge of the probabilities of those events. This is more like looking at a picture of the top of a die and seeing a six: what are the odds that it's a standard die, vs. a die with sixes on each face? I don't think it's meaningful to talk about that, or the sun going around the Earth, without using a Bayesian prior.

duneroadrunner 10y ago
> So I think there is a fundamental difference between "What are the odds of rolling a six" as "What are the odds of that event happening," and "What are the odds the Sun revolves around the Earth" as a question about whether or not something is true. Right, using Bayes makes the difference not a "fundamental" one, but just a practical one of coming up with the Bayesian prior. Even if it would be hard to establish a consensus on the most appropriate complete set of factors determining the Bayesian prior, there are clearly some examples of meaningful inputs. Like, for example, if you somehow had information about how many of the researcher's previous hypotheses on the subject failed to obtain a "p-value of significance". But perhaps more practically, you could consider things like the (Kolmogorov) complexity of the hypothesis. Since the number of "low complexity" hypotheses are finite, they are less susceptible to "p-value mining"[1]. The challenge being deciding which inputs to use to evaluate the complexity. This seems to me like an area where machine learning should be applicable. Rather than lament about the impracticality of determining an appropriate, tractable set of (quantifiable) criteria for determining a Bayesian prior, why not just include every potentially relevant piece of information and let Deep Thought[2] figure out which are actually relevant? So what we really need is unified data about all published results that have been confirmed and discredited. [1] obligatory xkcd: https://www.xkcd.com/882/ https://www.xkcd.com/882/ [2] for the youngsters: https://en.wikipedia.org/wiki/List_of_minor_The_Hitchhiker%27s_Guide_to_the_Galaxy_characters#Deep_Thought https://en.wikipedia.org/wiki/List_of_minor_The_Hitchhiker%2...

by bbctol 10y ago

duneroadrunner 10y ago

> So I think there is a fundamental difference between "What are the odds of rolling a six" as "What are the odds of that event happening," and "What are the odds the Sun revolves around the Earth" as a question about whether or not something is true. Right, using Bayes makes the difference not a "fundamental" one, but just a practical one of coming up with the Bayesian prior. Even if it would be hard to establish a consensus on the most appropriate complete set of factors determining the Bayesian prior, there are clearly some examples of meaningful inputs. Like, for example, if you somehow had information about how many of the researcher's previous hypotheses on the subject failed to obtain a "p-value of significance". But perhaps more practically, you could consider things like the (Kolmogorov) complexity of the hypothesis. Since the number of "low complexity" hypotheses are finite, they are less susceptible to "p-value mining"[1]. The challenge being deciding which inputs to use to evaluate the complexity. This seems to me like an area where machine learning should be applicable. Rather than lament about the impracticality of determining an appropriate, tractable set of (quantifiable) criteria for determining a Bayesian prior, why not just include every potentially relevant piece of information and let Deep Thought[2] figure out which are actually relevant? So what we really need is unified data about all published results that have been confirmed and discredited. [1] obligatory xkcd: https://www.xkcd.com/882/ https://www.xkcd.com/882/ [2] for the youngsters: https://en.wikipedia.org/wiki/List_of_minor_The_Hitchhiker%27s_Guide_to_the_Galaxy_characters#Deep_Thought https://en.wikipedia.org/wiki/List_of_minor_The_Hitchhiker%2...