4 ms·

This is what P-values are for

by jtmarmon 9y ago

shock 9y ago
Perhaps explain how P-values are to be used in this case, for the lay of us.
- baobrain 9y ago
  DISCLAIMER: EXTREME SIMPLIFICATION Given a statement and a sample with a mean and standard deviation, what is the probability that you manage to choose a sample with that mean and SD, assuming the statement is true? For example, my statement is the average weight of chicken an american eats every month is 20 pounds and my sample gives me a mean of 16 and SD of 4. The p value is the probability of choosing a group of people who eat on average 16 pounds of chicken every month if the REAL mean is actually 20. Edit to answer question: The p value doesn't "care" about the sample size - it is adjusted and remains accurate. And so if the p value if high for the sample in the study, that means it is a likely occurrence and the result is not very noteworthy.
- Shank 9y ago
  A p-value in a test statistic is the probability that the result of the test statistic could have been achieved through random data variations. A p-value is described in tandem with an alpha level, the minimum acceptable value to reject the null hypothesis in the test statistic. In short: typical alpha is 0.05, so a p-value < 0.05 is considered "statistically significant" and the results are not attributable to random chance in a t-test. An unpaired t-test compares two populations, and a p-value below the alpha level indicates that differences are not due to random chance. Take everything with a grain of salt, though. Low sample sizes dramatically change how test statistics produce results. P-value hacking is the practice of eliminating data from an experiment to produce statistically significant results or otherwise altering the data by adding buffers to ensure that a result confirms a belief a researcher has.
neaden 9y ago
Not really. I mean in an ideal world yes, but right now we have no idea how many other people did a similar study, found no difference, and just didn't publish anything. Let alone accidental or purposeful p-hacking.

This is what P-values are for

shock 9y ago

Perhaps explain how P-values are to be used in this case, for the lay of us.

baobrain 9y ago
DISCLAIMER: EXTREME SIMPLIFICATION Given a statement and a sample with a mean and standard deviation, what is the probability that you manage to choose a sample with that mean and SD, assuming the statement is true? For example, my statement is the average weight of chicken an american eats every month is 20 pounds and my sample gives me a mean of 16 and SD of 4. The p value is the probability of choosing a group of people who eat on average 16 pounds of chicken every month if the REAL mean is actually 20. Edit to answer question: The p value doesn't "care" about the sample size - it is adjusted and remains accurate. And so if the p value if high for the sample in the study, that means it is a likely occurrence and the result is not very noteworthy.
Shank 9y ago
A p-value in a test statistic is the probability that the result of the test statistic could have been achieved through random data variations. A p-value is described in tandem with an alpha level, the minimum acceptable value to reject the null hypothesis in the test statistic. In short: typical alpha is 0.05, so a p-value < 0.05 is considered "statistically significant" and the results are not attributable to random chance in a t-test. An unpaired t-test compares two populations, and a p-value below the alpha level indicates that differences are not due to random chance. Take everything with a grain of salt, though. Low sample sizes dramatically change how test statistics produce results. P-value hacking is the practice of eliminating data from an experiment to produce statistically significant results or otherwise altering the data by adding buffers to ensure that a result confirms a belief a researcher has.

neaden 9y ago

Not really. I mean in an ideal world yes, but right now we have no idea how many other people did a similar study, found no difference, and just didn't publish anything. Let alone accidental or purposeful p-hacking.