3 ms·

From the Journal Article: "They included a total of 550 students from Stanford's Bing Nursery School, aged about 4 years old (ranging from 2 to 6). Many of the

by acchow 5y ago

From the Journal Article: "They included a total of 550 students from Stanford's Bing Nursery School, aged about 4 years old (ranging from 2 to 6). Many of the participants are children of Stanford faculty and staff." https://www.sciencedirect.com/science/article/pii/S0167268119302641?via%3Dihub#bib0024 https://www.sciencedirect.com/science/article/pii/S016726811... How can we conclude anything at all about the general population using a sample of Stanford kids?

gkop 5y ago
This isn’t my field, but I think this is par for the course unfortunately and a manifestation of a larger issue. Eg https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0168354&type=printable https://journals.plos.org/plosone/article/file?id=10.1371/jo...
dahart 5y ago
I like to ask this whenever Dunning Kruger comes up; DK was a sample of Cornell undergrads, and the study was one tenth the size of the Stanford study. DK participants were volunteers of a psych class who got extra credit. Presumably they needed and could use extra credit, which may have excluded the A and the F students. It’s hard to imagine ways to start with more bias, or how we can possibly accept this sample as representative of humanity.
ren_engineer 5y ago
Social sciences are all a sham, now think about the trillions of dollars of government spending that are based on that same sham science as justification. People wonder why so many government programs fail, it's because they are built on a rotten foundation https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis >How can we conclude anything at all about the general population using a sample of Stanford kids? it's a well known problem that is rarely brought up, WEIRD bias. Most social science research participants are college students being bribed with extra credit or gift cards https://en.wikipedia.org/wiki/Psychology#WEIRD_bias https://en.wikipedia.org/wiki/Psychology#WEIRD_bias
- brimble 5y ago
  Good social science possible, but is difficult and (sometimes) expensive. If you can get the same personal outcome by doing something cheap and easy instead, of course that's what most people are going to do. Fixing that seems to be the Big Problem for most of science, for at least the last few decades (though, yes, particularly social science). > People wonder why so many government programs fail This, though, I'm not so sure about. Do government programs fail at a rate greater than those undertaken by other large organizations, like corporations or non-profits?
  - acchow 5y ago
    > Good social science possible, but is difficult and (sometimes) expensive The problem is the system incentivizes bad social science.
    - s1artibartfast 5y ago
      That and people overstate and over interpret things. There is nothing wrong with doing a study with faculty children. It is a great idea to get a bigger study done within a budget and timeframe. They write it up stating as much and their intelligent peers understand the caveats. Unfortunately, the results are so exciting that it starts getting cited and taught by people who leave off the caveats. This is why it is important for someone to go back and follow up on the original experiment with new studies.

by acchow 5y ago

gkop 5y ago

This isn’t my field, but I think this is par for the course unfortunately and a manifestation of a larger issue. Eg https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0168354&type=printable https://journals.plos.org/plosone/article/file?id=10.1371/jo...

dahart 5y ago

I like to ask this whenever Dunning Kruger comes up; DK was a sample of Cornell undergrads, and the study was one tenth the size of the Stanford study. DK participants were volunteers of a psych class who got extra credit. Presumably they needed and could use extra credit, which may have excluded the A and the F students. It’s hard to imagine ways to start with more bias, or how we can possibly accept this sample as representative of humanity.

ren_engineer 5y ago

Social sciences are all a sham, now think about the trillions of dollars of government spending that are based on that same sham science as justification. People wonder why so many government programs fail, it's because they are built on a rotten foundation https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis >How can we conclude anything at all about the general population using a sample of Stanford kids? it's a well known problem that is rarely brought up, WEIRD bias. Most social science research participants are college students being bribed with extra credit or gift cards https://en.wikipedia.org/wiki/Psychology#WEIRD_bias https://en.wikipedia.org/wiki/Psychology#WEIRD_bias

brimble 5y ago
Good social science possible, but is difficult and (sometimes) expensive. If you can get the same personal outcome by doing something cheap and easy instead, of course that's what most people are going to do. Fixing that seems to be the Big Problem for most of science, for at least the last few decades (though, yes, particularly social science). > People wonder why so many government programs fail This, though, I'm not so sure about. Do government programs fail at a rate greater than those undertaken by other large organizations, like corporations or non-profits?
- acchow 5y ago
  > Good social science possible, but is difficult and (sometimes) expensive The problem is the system incentivizes bad social science.
  - s1artibartfast 5y ago
    That and people overstate and over interpret things. There is nothing wrong with doing a study with faculty children. It is a great idea to get a bigger study done within a budget and timeframe. They write it up stating as much and their intelligent peers understand the caveats. Unfortunately, the results are so exciting that it starts getting cited and taught by people who leave off the caveats. This is why it is important for someone to go back and follow up on the original experiment with new studies.