3 ms·

> If the Dunning-Kruger effect were present, it would show up in Figure 11 as a downward trend in the data (similar to the trend in Figure 7). Such a trend woul

> If the Dunning-Kruger effect were present, it would show up in Figure 11 as a downward trend in the data (similar to the trend in Figure 7). Such a trend would indicate that unskilled people overestimate their ability, and that this overestimate decreases with skill. Looking at Figure 11, there is no hint of a trend. There certainly is a hint of a trend. Why do people, when visualizing data with a distinct trend, say that because the "error bars" from a particular statistical test overlap zero that no trend exists!? Freshman trend to over-confidence. Grad students trend to under-confidence. Undergrads in general trend to over-confidence (though this trend decreases as year in school increases), and post-graduates, whether grad students or professors, trend to under-confidence. These "trends" are not statistically significant, but they certainly are a trend! Also, the random data distribution in figure 9 doesn't show the same trends as Dunning-Kruger's curve in figure 2. Perhaps there is at least one psycho-social mechanism here worth investigating?

mrkeen 3y ago
> These "trends" are not statistically significant, but they certainly are a trend! This is an oxymoron.
- Dylan16807 3y ago
  Oxymorons only sound contradictory on a surface level. Something "certainly" being a "trend" is the definition of statistical significance, so this is a straight up contradiction.
  - anonymouskimmer 3y ago
    See here: https://news.ycombinator.com/item?id=38416858 https://news.ycombinator.com/item?id=38416858 "Trend" has multiple meanings. Statistics doesn't get to claim all of the meaning.
- anonymouskimmer 3y ago
  Show how. I place mechanistic theory prior to statistics in science. Mechanistic theory can be tested, statistics are a kind of test. If a statistically-insignificant result shows consistent, though non-significant deviations, such as the kind seen in Figure 11, then it tells me it's worth investigating whether mechanism(s) are explaining a very small portion of the variation that will not, in itself, show up as statistically significant, as it's being swamped by variation in other parameters.
  - Dylan16807 3y ago
    Consistency is a synonym for statistical significance. If there's consistency beyond random alignment, then there should be a statistical test you can apply over your data to extract the signal. You can extract surprisingly small signals relative to variation in other parameters. But if it's actually swamped, then it might not be real, so go get more data.
    - anonymouskimmer 3y ago
      > Consistency is a synonym for statistical significance. So basically you're telling me that if I can visually see a consistency that does not show up in their statistical test, then they aren't running an appropriate statistical test on what I'm seeing. > But if it's actually swamped, then it might not be real, so go get more data. Even better to design other experiments.
      - Dylan16807 3y ago
        > So basically you're telling me that if I can visually see a consistency that does not show up in their statistical test, then they aren't running an appropriate statistical test on what I'm seeing. Either they're not doing the right statistics, or it's a "consistency" that is much more likely to show up randomly than you naively expect, and the study needs to be repeated or enhanced. Sometimes you can see a pattern that's just a figment of chance. See also: numerology, jelly bean xkcd
Dylan16807 3y ago
If they're actually error bars, you can shrink them with more data. That will turn the hint of a trend into an observation of a trend. If it wasn't random noise giving a fake hint.
- anonymouskimmer 3y ago
  > If they're actually error bars, you can shrink them with more data. Assuming the new data has the same systemic or instrumental bias as the old data. Even using a different test date could skew results enough to widen the error bars.

mrkeen 3y ago

> These "trends" are not statistically significant, but they certainly are a trend! This is an oxymoron.

Dylan16807 3y ago
Oxymorons only sound contradictory on a surface level. Something "certainly" being a "trend" is the definition of statistical significance, so this is a straight up contradiction.
- anonymouskimmer 3y ago
  See here: https://news.ycombinator.com/item?id=38416858 https://news.ycombinator.com/item?id=38416858 "Trend" has multiple meanings. Statistics doesn't get to claim all of the meaning.
anonymouskimmer 3y ago
Show how. I place mechanistic theory prior to statistics in science. Mechanistic theory can be tested, statistics are a kind of test. If a statistically-insignificant result shows consistent, though non-significant deviations, such as the kind seen in Figure 11, then it tells me it's worth investigating whether mechanism(s) are explaining a very small portion of the variation that will not, in itself, show up as statistically significant, as it's being swamped by variation in other parameters.
- Dylan16807 3y ago
  Consistency is a synonym for statistical significance. If there's consistency beyond random alignment, then there should be a statistical test you can apply over your data to extract the signal. You can extract surprisingly small signals relative to variation in other parameters. But if it's actually swamped, then it might not be real, so go get more data.
  - anonymouskimmer 3y ago
    > Consistency is a synonym for statistical significance. So basically you're telling me that if I can visually see a consistency that does not show up in their statistical test, then they aren't running an appropriate statistical test on what I'm seeing. > But if it's actually swamped, then it might not be real, so go get more data. Even better to design other experiments.
    - Dylan16807 3y ago
      > So basically you're telling me that if I can visually see a consistency that does not show up in their statistical test, then they aren't running an appropriate statistical test on what I'm seeing. Either they're not doing the right statistics, or it's a "consistency" that is much more likely to show up randomly than you naively expect, and the study needs to be repeated or enhanced. Sometimes you can see a pattern that's just a figment of chance. See also: numerology, jelly bean xkcd

Dylan16807 3y ago

If they're actually error bars, you can shrink them with more data. That will turn the hint of a trend into an observation of a trend. If it wasn't random noise giving a fake hint.

anonymouskimmer 3y ago
> If they're actually error bars, you can shrink them with more data. Assuming the new data has the same systemic or instrumental bias as the old data. Even using a different test date could skew results enough to widen the error bars.