5 ms·

I still don't understand why we "believed" studies that claimed knowledge about typical behavior and subconscious decisions while those studies were only based

by Tarsul 5y ago

I still don't understand why we "believed" studies that claimed knowledge about typical behavior and subconscious decisions while those studies were only based on a single experiment with limited participants, e.g. Stanford Prison Experiment. These type of studies should ALWAYS be looked at critically and not just because they fail to reproduce but because they are based on a very small sample size in a very discreet scenario (and probably with participants who are not diverse).

camjohnson26 5y ago
Appeal to authority is a big reason. The academic leaders in the field were incentivized to find breakthroughs, or to treat any experiment they ran as a breakthrough, because that’s where their influence and reputation came from. People are so used to trusting experts that they didn’t realize that finding the truth wasn’t the primary motivator for some people.
enkid 5y ago
Tangent, but the Stanford Prison Experiment is an awful example of science. It was a researcher who wanted to prove a point and created the conditions to collect the data to prove that point. I hate that it's often the only psychological experiment many people are familiar with.
- seph-reed 5y ago
  Calling it an awful example of science seems a bit extreme. There is definitely something to learn from such an experiment, albeit not what was intended.
- mateo411 5y ago
  What about the Milgram experiment? This is also a very well known psychological experiment. Was the science behind the Milgram experiment rigorous?
  - enkid 5y ago
    From what I have read (I'm not a psychologist, so take everything I say with a grain of salt), the Milgram experiment had some major ethical issues and doesn't meet modern standards for statistical evidence. There's also accusations that the data may have been manipulated and the results don't directly support the claim that Milgram was making. For example, a large number of participants may have been aware that the actor wasn't really suffering when they administered the punishment. This would seriously skew the results.
  - jbullock35 5y ago
    Milgram's obedience studies didn't involve randomized treatments, and he had small numbers of subjects (typically around 40) in each of his many conditions. On the other hand, Milgram's investigations were serious, systematic, and in good faith -- which makes them worlds better than the Stanford Prison Experiment. Another reply in this thread suggests that "a large number of participants may have been aware that the actor wasn't really suffering when they administered the punishment." I've studied the topic and found no evidence of this point. In addition, the claim is hard to square with many subjects' reactions -- for example, their nervous laughter and their frequent protests, even as they continued to deliver what they thought were harmful electric shocks. See https://news.ycombinator.com/item?id=25928569 https://news.ycombinator.com/item?id=25928569 for more on efforts to replicate Milgram's results. Tl;dr: there is no equivalence between the Stanford Prison Experiment and Milgram's work on obedience. Milgram's work was superior.
- 99_00 5y ago
  >the Stanford Prison Experiment is an awful example of science. This may be true, but I don't think the evidence you give supports your assertion. >It was a researcher who wanted to prove a point and created the conditions to collect the data to prove that point "prove a point" is the hypothesis "created the conditions" is the experiment "collect the data to prove that point" is the observation
  - enkid 5y ago
    No, an experiment should not be set up to prove a hypothesis, it should be set up to test a hypothesis. In one the hypothesis is falsifiable, and in the other it's not.
    - 99_00 5y ago
      If people didn't act the way the experimenter expected wouldn't that disprove their hypothesis in this case?
      - enkid 5y ago
        No, because people did behave differently than he expected, and he coached them to behave in line with his expectations. He wasn't a neutral observer, he was a guiding force and the superintendent of the pretend prison. This is what I mean when I say it wasn't an experiment.
naasking 5y ago
The problem is not just with "belief", but in the process itself. Non-replicable studies are actually cited more than replicable studies: https://advances.sciencemag.org/content/7/21/eabd1705 https://advances.sciencemag.org/content/7/21/eabd1705 Science journalists probably are vulnerable to the same thing influences that lead scientists to do this, except they have even less review on their claims and so they become pop culture sound bites. As for why non-replicable results are cited more, I'd speculate that non-replicable results are often more unintuitive and surprising, and per the above link, reviewers apply lower standards on these papers in the hopes of finding something truly interesting and/or exciting. Not just in the results mind you, sometimes papers also apply a novel methodology that might be worth wider discussion. I'm not sure that's worth the reduction in credibility though.
- theptip 5y ago
  > As for why non-replicable results are cited more, I'd speculate that non-replicable results are often more unintuitive and surprising This seems the likely explanation; I saw a paper recently that showed that lay people can predict what will replicate with above-chance accuracy[1]. I imagine scientists are even better than lay people at this. So non-replicable results are almost by definition surprising (i.e. they are hypotheses that don't match our current model of how the world works), and surprising results are definitely better news than unsurprising results. [1](https://journals.sagepub.com/doi/full/10.1177/2515245920919667 https://journals.sagepub.com/doi/full/10.1177/25152459209196...)
- edanm 5y ago
  > As for why non-replicable results are cited more, I'd speculate that non-replicable results are often more unintuitive and surprising I mean, a kind of well known thing is that in general, something false can be more interesting than something true, since it has more degrees of freedom. You can make up anything you want - the truth has to conform to what's actually real.
TaupeRanger 5y ago
Same reason we believed eggs will give you heart disease. Science reporting is terrible and the general education system doesn't teach rational skepticism, it teaches unconditional trust of intellectual authority.
- reedjosh 5y ago
  Indeed! And in reality eggs are basically a superfood.
- denimnerd42 5y ago
  every mainstream thought about diet is untrustworthy to me due to the above type of example :/
  - loudtieblahblah 5y ago
    it is. mainstream diets will tell you: * saturated fats are bad (lies told us by a crappy Ancel Keys study, promoted for decades by processed food companies (like Kellogs) ran by Seventh Day Adventists who were convinced "meat led man to dangerous impulses and temptation") * polyunsaturated fats are good. The American Heart Assocation had an article up for years that went as far as to claim Omega6s are heart healthy. They only recently took it down this year. But we know they're inflammatory and we know we're consuming 25-100x more Omega6s than we ever would before the industrial invention of seed-oils being shoved into every product imaginable (bread, cereal, granola, anything that comes in a box, feed given to animals meant for meat production) here's a webmd article on it: https://www.webmd.com/heart/news/20090126/expert-panel-omega-6-wont-hurt-heart https://www.webmd.com/heart/news/20090126/expert-panel-omega... \ * sunlight creates high cancer risk (ignoring that cancer is unlikely, treatment if caught early has a high survival rate, and the risk of not having vitamin d throughout your life risks far more likely autoimmune issues, depression, anxiety and even certain cancers and inflammatory disease). * sugar is good for you. Sure, they'll specify processed sugars are bad for you or "added sugar", but common wisdom will accept a NET (subtract fiber) 200-300g carb diet as acceptable. Grain is still often listed as the most important and largest part of the food pyramid. The reality is - all mainstream health advice, including that which you'll get from your doctor who got a whole single nutrition class in school, ensures that processed foods don't loose business on the front end and the medical/pharma industries don't lose money on the back end. even in the push for a more vegetarian/blue-zone diet world - they're doing so by promoting meat alternatives like "Beyond Meat" which is chock full of so much seed oil and other processed substances, it's mainstreaming vegetarianism-as-fast-food. McDonald's burger.. is still a McDonald's burger and you shouldn't be eating it. If a factory isn't making it at scale, shoving it in a box, branding it and ensuring you don't have to spend any time making/cooking/preparing whole, fresh foods (those pesky things that tend to have short shelf lives and are costly to Ag businesses), then your PCP, the government, most food businesses, your medical insurance company, absolutely no one of any kind of "authority" isn't going to promote it highly. They'll do ANYTHING except remove seed oils. They'll make your potato chips out of broccoli and carrots and still drench them in sunflower or canola oil. They'll reduce the salt. they'll make shit out of beets. And still manage to make it horrible for you. The MSM regurgitates "health" info regarding diets in a way that acts as advertising for these orgs.
batguano 5y ago
I assume you are referring to "The Stanford Prison Anecdote"?
hpoe 5y ago
They are believed because they are sensational, they are titillating, and it gives people the excuse for their bad behavior by believing that everyone would be a monster in the right circumstances.
Blikkentrekker 5y ago
Who is “we”? This has been criticized since 1999 and far longer back. How long ago is the Rosenhan Experiment again? Dare I say that a majority have always held a dismissive, critical view of such matters. But of course, those that hold dismissive views of it are not the ones who work in such fields, and certainly not at the top ready to implement changes, so it can continue to persist and go on despite of being highly criticized. At least when I studied physics at a university around what must have been 2005, most of the students and professors there were highly critical of softer science and it often came up that some of these papers popped up and were viciously criticized for clear and obvious systematic errors in the methodology.
loudtieblahblah 5y ago
same reason we believe observational studies in nutrition, or worse - studies on animals. if you have the weight of peer review or at least a well documented study, then the media runs wild with it's claims, it gets shoved into textbooks, then governments shape policy on those claims, corporations and medical practices sell gimmicks, books, supplements, therapy and plans of action to heal you... it all becomes lies, half-truths, bad data all just repeating itself ad naseum until "truth" is established in the public consciousness. Quacks on the web, the American Heart Association, your local doctor's office will all pedal garbage based on the bad data. And once it's well established as true, backing away from it is hard because it's become so woven, institutionally. This is why people think saturated fat and sunlight are bad or at least a net-negative. Even modern medicine, psychology and nutrition sciences all have horrible replication crisis's and we're no better in rejecting the nonsense now than we were then.
aaron695 5y ago
SPE was needed at the time because everyone knew the Germans were like us, but needed a framework to say it. It also opened up the idea the Japanese were like us too. It allowed us to say what we believed and build on that. It's not science. But science isn't the only way forward.
- acchow 5y ago
  Can't we just look at history and conclude that we are "just like the Germans"? Why do we need to shroud it in pseudo-science?

by Tarsul 5y ago

camjohnson26 5y ago

Appeal to authority is a big reason. The academic leaders in the field were incentivized to find breakthroughs, or to treat any experiment they ran as a breakthrough, because that’s where their influence and reputation came from. People are so used to trusting experts that they didn’t realize that finding the truth wasn’t the primary motivator for some people.

enkid 5y ago

Tangent, but the Stanford Prison Experiment is an awful example of science. It was a researcher who wanted to prove a point and created the conditions to collect the data to prove that point. I hate that it's often the only psychological experiment many people are familiar with.

seph-reed 5y ago
Calling it an awful example of science seems a bit extreme. There is definitely something to learn from such an experiment, albeit not what was intended.
mateo411 5y ago
What about the Milgram experiment? This is also a very well known psychological experiment. Was the science behind the Milgram experiment rigorous?
- enkid 5y ago
  From what I have read (I'm not a psychologist, so take everything I say with a grain of salt), the Milgram experiment had some major ethical issues and doesn't meet modern standards for statistical evidence. There's also accusations that the data may have been manipulated and the results don't directly support the claim that Milgram was making. For example, a large number of participants may have been aware that the actor wasn't really suffering when they administered the punishment. This would seriously skew the results.
- jbullock35 5y ago
  Milgram's obedience studies didn't involve randomized treatments, and he had small numbers of subjects (typically around 40) in each of his many conditions. On the other hand, Milgram's investigations were serious, systematic, and in good faith -- which makes them worlds better than the Stanford Prison Experiment. Another reply in this thread suggests that "a large number of participants may have been aware that the actor wasn't really suffering when they administered the punishment." I've studied the topic and found no evidence of this point. In addition, the claim is hard to square with many subjects' reactions -- for example, their nervous laughter and their frequent protests, even as they continued to deliver what they thought were harmful electric shocks. See https://news.ycombinator.com/item?id=25928569 https://news.ycombinator.com/item?id=25928569 for more on efforts to replicate Milgram's results. Tl;dr: there is no equivalence between the Stanford Prison Experiment and Milgram's work on obedience. Milgram's work was superior.
99_00 5y ago
>the Stanford Prison Experiment is an awful example of science. This may be true, but I don't think the evidence you give supports your assertion. >It was a researcher who wanted to prove a point and created the conditions to collect the data to prove that point "prove a point" is the hypothesis "created the conditions" is the experiment "collect the data to prove that point" is the observation
- enkid 5y ago
  No, an experiment should not be set up to prove a hypothesis, it should be set up to test a hypothesis. In one the hypothesis is falsifiable, and in the other it's not.
  - 99_00 5y ago
    If people didn't act the way the experimenter expected wouldn't that disprove their hypothesis in this case?
    - enkid 5y ago
      No, because people did behave differently than he expected, and he coached them to behave in line with his expectations. He wasn't a neutral observer, he was a guiding force and the superintendent of the pretend prison. This is what I mean when I say it wasn't an experiment.

naasking 5y ago

The problem is not just with "belief", but in the process itself. Non-replicable studies are actually cited more than replicable studies: https://advances.sciencemag.org/content/7/21/eabd1705 https://advances.sciencemag.org/content/7/21/eabd1705 Science journalists probably are vulnerable to the same thing influences that lead scientists to do this, except they have even less review on their claims and so they become pop culture sound bites. As for why non-replicable results are cited more, I'd speculate that non-replicable results are often more unintuitive and surprising, and per the above link, reviewers apply lower standards on these papers in the hopes of finding something truly interesting and/or exciting. Not just in the results mind you, sometimes papers also apply a novel methodology that might be worth wider discussion. I'm not sure that's worth the reduction in credibility though.

theptip 5y ago
> As for why non-replicable results are cited more, I'd speculate that non-replicable results are often more unintuitive and surprising This seems the likely explanation; I saw a paper recently that showed that lay people can predict what will replicate with above-chance accuracy[1]. I imagine scientists are even better than lay people at this. So non-replicable results are almost by definition surprising (i.e. they are hypotheses that don't match our current model of how the world works), and surprising results are definitely better news than unsurprising results. [1](https://journals.sagepub.com/doi/full/10.1177/2515245920919667 https://journals.sagepub.com/doi/full/10.1177/25152459209196...)
edanm 5y ago
> As for why non-replicable results are cited more, I'd speculate that non-replicable results are often more unintuitive and surprising I mean, a kind of well known thing is that in general, something false can be more interesting than something true, since it has more degrees of freedom. You can make up anything you want - the truth has to conform to what's actually real.

TaupeRanger 5y ago

Same reason we believed eggs will give you heart disease. Science reporting is terrible and the general education system doesn't teach rational skepticism, it teaches unconditional trust of intellectual authority.

reedjosh 5y ago
Indeed! And in reality eggs are basically a superfood.
denimnerd42 5y ago
every mainstream thought about diet is untrustworthy to me due to the above type of example :/
- loudtieblahblah 5y ago
  it is. mainstream diets will tell you: * saturated fats are bad (lies told us by a crappy Ancel Keys study, promoted for decades by processed food companies (like Kellogs) ran by Seventh Day Adventists who were convinced "meat led man to dangerous impulses and temptation") * polyunsaturated fats are good. The American Heart Assocation had an article up for years that went as far as to claim Omega6s are heart healthy. They only recently took it down this year. But we know they're inflammatory and we know we're consuming 25-100x more Omega6s than we ever would before the industrial invention of seed-oils being shoved into every product imaginable (bread, cereal, granola, anything that comes in a box, feed given to animals meant for meat production) here's a webmd article on it: https://www.webmd.com/heart/news/20090126/expert-panel-omega-6-wont-hurt-heart https://www.webmd.com/heart/news/20090126/expert-panel-omega... \ * sunlight creates high cancer risk (ignoring that cancer is unlikely, treatment if caught early has a high survival rate, and the risk of not having vitamin d throughout your life risks far more likely autoimmune issues, depression, anxiety and even certain cancers and inflammatory disease). * sugar is good for you. Sure, they'll specify processed sugars are bad for you or "added sugar", but common wisdom will accept a NET (subtract fiber) 200-300g carb diet as acceptable. Grain is still often listed as the most important and largest part of the food pyramid. The reality is - all mainstream health advice, including that which you'll get from your doctor who got a whole single nutrition class in school, ensures that processed foods don't loose business on the front end and the medical/pharma industries don't lose money on the back end. even in the push for a more vegetarian/blue-zone diet world - they're doing so by promoting meat alternatives like "Beyond Meat" which is chock full of so much seed oil and other processed substances, it's mainstreaming vegetarianism-as-fast-food. McDonald's burger.. is still a McDonald's burger and you shouldn't be eating it. If a factory isn't making it at scale, shoving it in a box, branding it and ensuring you don't have to spend any time making/cooking/preparing whole, fresh foods (those pesky things that tend to have short shelf lives and are costly to Ag businesses), then your PCP, the government, most food businesses, your medical insurance company, absolutely no one of any kind of "authority" isn't going to promote it highly. They'll do ANYTHING except remove seed oils. They'll make your potato chips out of broccoli and carrots and still drench them in sunflower or canola oil. They'll reduce the salt. they'll make shit out of beets. And still manage to make it horrible for you. The MSM regurgitates "health" info regarding diets in a way that acts as advertising for these orgs.

batguano 5y ago

I assume you are referring to "The Stanford Prison Anecdote"?

hpoe 5y ago

They are believed because they are sensational, they are titillating, and it gives people the excuse for their bad behavior by believing that everyone would be a monster in the right circumstances.

Blikkentrekker 5y ago

Who is “we”? This has been criticized since 1999 and far longer back. How long ago is the Rosenhan Experiment again? Dare I say that a majority have always held a dismissive, critical view of such matters. But of course, those that hold dismissive views of it are not the ones who work in such fields, and certainly not at the top ready to implement changes, so it can continue to persist and go on despite of being highly criticized. At least when I studied physics at a university around what must have been 2005, most of the students and professors there were highly critical of softer science and it often came up that some of these papers popped up and were viciously criticized for clear and obvious systematic errors in the methodology.

loudtieblahblah 5y ago

same reason we believe observational studies in nutrition, or worse - studies on animals. if you have the weight of peer review or at least a well documented study, then the media runs wild with it's claims, it gets shoved into textbooks, then governments shape policy on those claims, corporations and medical practices sell gimmicks, books, supplements, therapy and plans of action to heal you... it all becomes lies, half-truths, bad data all just repeating itself ad naseum until "truth" is established in the public consciousness. Quacks on the web, the American Heart Association, your local doctor's office will all pedal garbage based on the bad data. And once it's well established as true, backing away from it is hard because it's become so woven, institutionally. This is why people think saturated fat and sunlight are bad or at least a net-negative. Even modern medicine, psychology and nutrition sciences all have horrible replication crisis's and we're no better in rejecting the nonsense now than we were then.

aaron695 5y ago

SPE was needed at the time because everyone knew the Germans were like us, but needed a framework to say it. It also opened up the idea the Japanese were like us too. It allowed us to say what we believed and build on that. It's not science. But science isn't the only way forward.

acchow 5y ago
Can't we just look at history and conclude that we are "just like the Germans"? Why do we need to shroud it in pseudo-science?