4 ms·

i immediately jump to think that replacing all the data with noise is a pretty good null hypothesis (at least for the analysis). is that not true?

by a-dub 5y ago

rawgabbit 5y ago
You're changing the definition of the null hypothesis. What you are essentially saying is that there is no need to perform experiments and studies, I can arbitrarily grab data from anywhere and if it fails to support the hypothesis... then the hypothesis is wrong.
- a-dub 5y ago
  if your analysis produces an effect when fed uniform random data, then yes, no need to perform experiments or studies, because all your results are null. right? so the hypothesis would be something like "the analysis shows a relationship between the two variables" and the null hypothesis would be something like "the analysis shows a relationship between two uniform random variables" and in this case that null is shown and accepted because no such relationship exists by definition. right? (unless it's like, "they have the same entropy", or something) i'm very rusty with this stuff, so clarification would be much appreciated!
  - a-dub 5y ago
    note that i have not carefully been through the claimed analysis here (and specifically have doubts about the data v. error plot) but if the claim that the analysis produces the effect with random inputs is assumed, then the whole thing can be rejected at step 0, right?
  - rawgabbit 5y ago
    In the case of DK, the hypothesis is that there is a bizarre relation between perceived skill and actual skill. The null hypothesis is that is there no relation or correlation between perceived skill and actual skill. A researcher would perform an experiment. If researcher observes statistically significant results. Then the researcher can reject the null hypothesis, and say the theory is valid. If researcher does not observe statistically significant results. They can only say their experiment doesn't support the theory.

i immediately jump to think that replacing all the data with noise is a pretty good null hypothesis (at least for the analysis). is that not true?

by a-dub 5y ago

i immediately jump to think that replacing all the data with noise is a pretty good null hypothesis (at least for the analysis). is that not true?

rawgabbit 5y ago

You're changing the definition of the null hypothesis. What you are essentially saying is that there is no need to perform experiments and studies, I can arbitrarily grab data from anywhere and if it fails to support the hypothesis... then the hypothesis is wrong.

a-dub 5y ago
if your analysis produces an effect when fed uniform random data, then yes, no need to perform experiments or studies, because all your results are null. right? so the hypothesis would be something like "the analysis shows a relationship between the two variables" and the null hypothesis would be something like "the analysis shows a relationship between two uniform random variables" and in this case that null is shown and accepted because no such relationship exists by definition. right? (unless it's like, "they have the same entropy", or something) i'm very rusty with this stuff, so clarification would be much appreciated!
- a-dub 5y ago
  note that i have not carefully been through the claimed analysis here (and specifically have doubts about the data v. error plot) but if the claim that the analysis produces the effect with random inputs is assumed, then the whole thing can be rejected at step 0, right?
- rawgabbit 5y ago
  In the case of DK, the hypothesis is that there is a bizarre relation between perceived skill and actual skill. The null hypothesis is that is there no relation or correlation between perceived skill and actual skill. A researcher would perform an experiment. If researcher observes statistically significant results. Then the researcher can reject the null hypothesis, and say the theory is valid. If researcher does not observe statistically significant results. They can only say their experiment doesn't support the theory.