3 ms·

It's ironic that this ... rant? ... is basically unreadable without knowledge of basic statistical methods. How do you teach any of this to someone who hasn't

It's ironic that this ... rant? ... is basically unreadable without knowledge of basic statistical methods. How do you teach any of this to someone who hasn't already taken introductory statistics? How do you learn anything if you first have to learn the myriad ways something you don't even have a basic working knowledge of can fail before you learn it?

mjburgess 3y ago
The comment is addressed to the informed reader who is the only one with a hope of being persuaded on this point. To teach this, from scratch, I think is fairly easy -- but there's few with any incentive to do it. Many in academia wouldnt know how, and if they did, would discover that much of their research can be shown a priori to not be worthwhile (rather than after a decade of 'debate'). All you really need is to start with establishing an intuitive understanding of randomness, how apparently highly patterned it is, and so on. Then ask: how easy is it to reproduce an observed pattern with (simulated) randomness? That question alone, properly supported via basic programming simulations, will take you extremely far. Indeed, the answer to it is often obvious -- a trivial program. That few ever write such programs shows how the whole edifice of stats education is geared towards confirmation bias. Before computers, stats was either an extremely mathematical disipline seeking (empirically useless) formula for toy models; or using heuristic empirical formula that rarely applied. Computers basically obviate all of that. Stats is mostly about counting things and making comparisons -- perfect tasks for machines. with only a few high-school mathematical formula most could derive most useful statistical techniques as simple computer programs.
- noelwelsh 3y ago
  The modern approach, of which this textbook is an example, does start with simulation. In fact there is very little classical statistics (distributions, analytic tests) in the book. The Berkeley Data 8 book, which I link to in another comment, takes the same approach. I imagine there is still too much classical material for your tastes, but there is definitely change happening.
- 2devnull 3y ago
  “ that much of their research can be shown a priori to not be worthwhile” Bingo. Cargo cult stats all the way down. It’s not just personal interest, it’s the entire field, it’s their colleagues, mentors, and students. Good luck getting somebody to see the light when not just their own income depends on not seeing it, their whole world depends on the “stat recipes” handed down from granny.
  - brutusborn 3y ago
    I think the egotistical aspect is the most powerful: many researchers have built an identity based on the fact that they “know” something, so to propose better alternatives to their pet theories is tantamount to proposing their life is a lie. To change their mind they need to admit they didn’t “know”. The better the alternatives, the more fierce the passion with which they will be rejected by the mainstream.
    - 2devnull 3y ago
      I now think it’s best explained by simple economics. Academia and academics are the product of economic forces by and large. It’s not quirky personalities or uniquely talented minds that make up academia today. It’s droves of conscientious (big five sense) conformists, with either high iq or mere socio-economic privilege, who have been trained by our society to feel that financial security means college, and even more financial security means even more college. Credentials are like alpha .05, they solve a scale problem in a way that alters the quality/quantity ratio. If you want more researchers/research/science output, credentials and alpha .05 cargo cult stats are your levers to get more quantity at lower quality.
Retric 3y ago
It seems like a reasonable critique. The suggestion is to include such ideas as people are taking introductory statistics which isn’t inappropriate. I wouldn’t suggest forcing students to code up their own simulations from scratch, but creating a framework where students can plug in various formula for each population, attach a statistical test, and then run various simulations could do quite a bit. However, what kinds of formula students are told to plug in are important. If every formula is producing bell curves then that’s a failure to educate people. 50d6 vs 50d6 + 1 is easy enough you can include 1d2 * 50 + 50d6 for a 2 tailed distribution, but also significantly different distributions which then fail various tests etc. I’ve seen people correctly remember the formula for statistical tests from memory and then wildly misapply them. That seems like focusing on the wrong things in an age when such information is at everyone’s fingertips, but understanding of what that information means isn’t.

mjburgess 3y ago

The comment is addressed to the informed reader who is the only one with a hope of being persuaded on this point. To teach this, from scratch, I think is fairly easy -- but there's few with any incentive to do it. Many in academia wouldnt know how, and if they did, would discover that much of their research can be shown a priori to not be worthwhile (rather than after a decade of 'debate'). All you really need is to start with establishing an intuitive understanding of randomness, how apparently highly patterned it is, and so on. Then ask: how easy is it to reproduce an observed pattern with (simulated) randomness? That question alone, properly supported via basic programming simulations, will take you extremely far. Indeed, the answer to it is often obvious -- a trivial program. That few ever write such programs shows how the whole edifice of stats education is geared towards confirmation bias. Before computers, stats was either an extremely mathematical disipline seeking (empirically useless) formula for toy models; or using heuristic empirical formula that rarely applied. Computers basically obviate all of that. Stats is mostly about counting things and making comparisons -- perfect tasks for machines. with only a few high-school mathematical formula most could derive most useful statistical techniques as simple computer programs.

noelwelsh 3y ago
The modern approach, of which this textbook is an example, does start with simulation. In fact there is very little classical statistics (distributions, analytic tests) in the book. The Berkeley Data 8 book, which I link to in another comment, takes the same approach. I imagine there is still too much classical material for your tastes, but there is definitely change happening.
2devnull 3y ago
“ that much of their research can be shown a priori to not be worthwhile” Bingo. Cargo cult stats all the way down. It’s not just personal interest, it’s the entire field, it’s their colleagues, mentors, and students. Good luck getting somebody to see the light when not just their own income depends on not seeing it, their whole world depends on the “stat recipes” handed down from granny.
- brutusborn 3y ago
  I think the egotistical aspect is the most powerful: many researchers have built an identity based on the fact that they “know” something, so to propose better alternatives to their pet theories is tantamount to proposing their life is a lie. To change their mind they need to admit they didn’t “know”. The better the alternatives, the more fierce the passion with which they will be rejected by the mainstream.
  - 2devnull 3y ago
    I now think it’s best explained by simple economics. Academia and academics are the product of economic forces by and large. It’s not quirky personalities or uniquely talented minds that make up academia today. It’s droves of conscientious (big five sense) conformists, with either high iq or mere socio-economic privilege, who have been trained by our society to feel that financial security means college, and even more financial security means even more college. Credentials are like alpha .05, they solve a scale problem in a way that alters the quality/quantity ratio. If you want more researchers/research/science output, credentials and alpha .05 cargo cult stats are your levers to get more quantity at lower quality.

Retric 3y ago

It seems like a reasonable critique. The suggestion is to include such ideas as people are taking introductory statistics which isn’t inappropriate. I wouldn’t suggest forcing students to code up their own simulations from scratch, but creating a framework where students can plug in various formula for each population, attach a statistical test, and then run various simulations could do quite a bit. However, what kinds of formula students are told to plug in are important. If every formula is producing bell curves then that’s a failure to educate people. 50d6 vs 50d6 + 1 is easy enough you can include 1d2 * 50 + 50d6 for a 2 tailed distribution, but also significantly different distributions which then fail various tests etc. I’ve seen people correctly remember the formula for statistical tests from memory and then wildly misapply them. That seems like focusing on the wrong things in an age when such information is at everyone’s fingertips, but understanding of what that information means isn’t.