3 ms·

I don't know any stats. I want to understand how to interpret p-values and it seems like according to this post I also need to know how to estimate the "Proport

by zawerf 8y ago

I don't know any stats. I want to understand how to interpret p-values and it seems like according to this post I also need to know how to estimate the "Proportion of True Hypotheses"? He starts off with "Imagine a world of scientific hypotheses" but isn't that world infinite? I could ask whether 1 is a number, 2 is a number, 3.14 is a chicken, etc. Aside: I would argue that the cardinality of those infinites are actually the same since you can form a bijection between true and false hypotheses just by negating the statement (1 is a giraffe, 1 is not a giraffe). So I assume he means the finite world of all hypothesis that scientists(which are finite since the universe will eventually end) would consider interesting enough to test? Even then it doesn't seem narrow enough. The pool of hypotheses has a shifting distribution based on time period (e.g. 1800s vs modern science). And you can further slice them down field, journal, affiliation, exact subject, etc. If that's being pedantic, ignore defining what the world of hypotheses looks like for now. Even given a fixed world, how can you estimate the true number of "True Hypotheses"? You only really see the end result of your study (positive or negative) and you probably don't even know about the existence of most of the negative studies. In short, still super confused! Links to intro to p-values/stats appreciated.

tscs37 8y ago
From what I understood in my recent statistics course, the p-value is the probability of receiving a specific result considering the null hypothesis is true. It's usually compared by using some distribution model (binomial, etc.) over the actual study results. Short version; the p-value is only sufficient to disprove a hypothesis, not to prove it. If your p-value is 1% than that means you make experiments until the probability that you are wrong is less than 1%. (for example, you throw coins 100 times, 90 times is head, the probability of that result is below 0.04 so the coin is not fair) That still doesn't mean you are right. Just that at the moment you have not been prove wrong or the data is not sufficient to prove you wrong.
jononor 8y ago
You don't need to know a universal "Proportion of True Hypotheses", that was just given as an example. But you do need to assign some probability to the hypothesis being true. Which is of course difficult to do with high certainty. But hopefully possible to argue whether it is closer to 0.1, 0.5 or 0.9 - and adjust interpretation of the pvalue accordingly.

by zawerf 8y ago

tscs37 8y ago

From what I understood in my recent statistics course, the p-value is the probability of receiving a specific result considering the null hypothesis is true. It's usually compared by using some distribution model (binomial, etc.) over the actual study results. Short version; the p-value is only sufficient to disprove a hypothesis, not to prove it. If your p-value is 1% than that means you make experiments until the probability that you are wrong is less than 1%. (for example, you throw coins 100 times, 90 times is head, the probability of that result is below 0.04 so the coin is not fair) That still doesn't mean you are right. Just that at the moment you have not been prove wrong or the data is not sufficient to prove you wrong.

jononor 8y ago

You don't need to know a universal "Proportion of True Hypotheses", that was just given as an example. But you do need to assign some probability to the hypothesis being true. Which is of course difficult to do with high certainty. But hopefully possible to argue whether it is closer to 0.1, 0.5 or 0.9 - and adjust interpretation of the pvalue accordingly.