3 ms·

Do yourself a favor and read the whole article. The author does a fantastic job of summing up the big issues facing psychology and a lot of my own thoughts reg

Do yourself a favor and read the whole article. The author does a fantastic job of summing up the big issues facing psychology and a lot of my own thoughts regarding the academic side of the social sciences in general. So very few people in the field seem to care about the basics of the scientific method (reproducible experiments, hypotheses that can be falsified, etc.) or of producing research that can really be applied in useful ways. And unfortunately so much of it seems to bleed over into pop science and business "coaching" these days, too.

jschveibinz 3y ago
Re: lack of scientific method, education is another psych-related field with the same problem. Try reading a few of the foundational or trendy education articles: the articles that actually affect how teachers are taught to educate kids. Then look at the methods used in these articles, and realize that some of them have sample sizes of 20 or 30 children. In other words, the conclusions are useless.
- whatshisface 3y ago
  A sample size of thirty could be sufficient if the effect being tested for is strong enough.
  - tomjakubowski 3y ago
    depends how you're sampling the population, doesn't it? if the sample is drawn only from, say, students from the same class at a particular boarding school, how likely are the effects to apply to the entire population of students?
    - whatshisface 3y ago
      It depends on what the effect is and the hypotheses you are attempting to distinguish between. Sometimes you have to use your brain and think about it; science is not all randomized controlled trials, p values and curve fitting. :)
      - ChainOfFools 3y ago
        I know this is not the way it's done on HN, but since this isn't my thread I feel like I can throw in some amicus support for your side here by pointing out that an N of 30 is a sort of foundational number in stats, although it's not a magical number as there are some pretty rigorous maths behind it. But it's not like social science people decided that an n of 20 or 30 was a good number to use because that's conveniently what they likely had access to (roughly the size of an undergrad classroom) for field data. and they didn't come up with it anyway), this is a number you're going to find in chapter 1, page 3 or 4 of any introductory stats text.
- dmvdoug 3y ago
  Fear not. Teachers, even baby teachers early in their career, very quickly learn the utter emptiness of “evidence-based practices” and “research-supported high-impact instructional strategies.” I have honestly not met a single damned teacher, including ones who lead professional development sessions, who believes that you can actually learn anything about real, day-to-day teaching by reading the shit education researchers produce. Even people who use educational research to justify their teaching methods usually only do so to satisfy an administrator or district-level person who needs to check some box mandated by the state or federal government to be checked. Among teachers, the real conversations are about adapting methods or strategies in order to make them realistically applicable. Or, honestly, how you can describe your instructional methods in such a way that they seem to be “supported by research.” Actual teaching is very much a craft. Justifying what you have found through experience to actually work with students is just an exercise in bullshitting.
  - esperent 3y ago
    > I have honestly not met a single damned teacher, including ones who lead professional development sessions, who believes that you can actually learn anything about real, day-to-day teaching by reading the shit education researchers produce Why do you think the teachers you have met are a representative sample of all teachers everywhere?
    - dmvdoug 3y ago
      Me: “Here’s my experience so far.” You: “Your statistical methods are flawed. Me: “???”
benterix 3y ago
In this particular case there is still something else: the author doing a series of quite similar studies with relatively little impact so that even if everything they did turns out fake, nobody cares. It's the exact opposite of the conclusion of the famous speech "You and Your Research".

jschveibinz 3y ago

Re: lack of scientific method, education is another psych-related field with the same problem. Try reading a few of the foundational or trendy education articles: the articles that actually affect how teachers are taught to educate kids. Then look at the methods used in these articles, and realize that some of them have sample sizes of 20 or 30 children. In other words, the conclusions are useless.

whatshisface 3y ago
A sample size of thirty could be sufficient if the effect being tested for is strong enough.
- tomjakubowski 3y ago
  depends how you're sampling the population, doesn't it? if the sample is drawn only from, say, students from the same class at a particular boarding school, how likely are the effects to apply to the entire population of students?
  - whatshisface 3y ago
    It depends on what the effect is and the hypotheses you are attempting to distinguish between. Sometimes you have to use your brain and think about it; science is not all randomized controlled trials, p values and curve fitting. :)
    - ChainOfFools 3y ago
      I know this is not the way it's done on HN, but since this isn't my thread I feel like I can throw in some amicus support for your side here by pointing out that an N of 30 is a sort of foundational number in stats, although it's not a magical number as there are some pretty rigorous maths behind it. But it's not like social science people decided that an n of 20 or 30 was a good number to use because that's conveniently what they likely had access to (roughly the size of an undergrad classroom) for field data. and they didn't come up with it anyway), this is a number you're going to find in chapter 1, page 3 or 4 of any introductory stats text.
dmvdoug 3y ago
Fear not. Teachers, even baby teachers early in their career, very quickly learn the utter emptiness of “evidence-based practices” and “research-supported high-impact instructional strategies.” I have honestly not met a single damned teacher, including ones who lead professional development sessions, who believes that you can actually learn anything about real, day-to-day teaching by reading the shit education researchers produce. Even people who use educational research to justify their teaching methods usually only do so to satisfy an administrator or district-level person who needs to check some box mandated by the state or federal government to be checked. Among teachers, the real conversations are about adapting methods or strategies in order to make them realistically applicable. Or, honestly, how you can describe your instructional methods in such a way that they seem to be “supported by research.” Actual teaching is very much a craft. Justifying what you have found through experience to actually work with students is just an exercise in bullshitting.
- esperent 3y ago
  > I have honestly not met a single damned teacher, including ones who lead professional development sessions, who believes that you can actually learn anything about real, day-to-day teaching by reading the shit education researchers produce Why do you think the teachers you have met are a representative sample of all teachers everywhere?
  - dmvdoug 3y ago
    Me: “Here’s my experience so far.” You: “Your statistical methods are flawed. Me: “???”

benterix 3y ago

In this particular case there is still something else: the author doing a series of quite similar studies with relatively little impact so that even if everything they did turns out fake, nobody cares. It's the exact opposite of the conclusion of the famous speech "You and Your Research".