19 ms·

Human psychology and behavioral studies overlook 85 percent of people

245 points by headalgorithm 8y ago

bellerose 8y ago
Locus of control is another way of understanding how humans are not necessarily universal to one another by how events are perceived. The missing shape example might just be the children never encounter the square & triangles in daily life and are thrown off it. I don't really consider that a great example for showing how behavioural is different.
- feanaro 8y ago
  The very fact that familiarity with a shape can influence performance on pattern recognition tasks so dramatically is a significant finding on its own. It would mean we are effectively blind(er) to unfamiliar shapes, even though they are extremely simple, like a triangle or square are.
  - bellerose 8y ago
    I’m surprised that is a recent discovery. I would think if someone asked a group of kids why someone would get the problem wrong. They would be like well first time seeing shapes.
    - feanaro 8y ago
      Well, some certainly do, but as always, it is a game of sampling, so individual errors are not as important as proportions. If we take the article at face value, proportions differ drastically between very different cultures so there must be something in the culture that biases the proportions towards one outcome or the other systematically. Of course, it's always possible any individual study contained unknown and unaccounted biases. Funding in science is still relatively small, the number of people participating few and science itself is hard.
  - KoenDG 8y ago
    I was just thinking. If they gave me this test back in the '90s, but replaced it with a Pokeball and the face of a character from the Pokemon show, I'd probably have done better too. Those would have been things recognizable to me. Things I liked a lot. Also, they would have put my mind somewhat at ease, I think. I wonder how much of the inability to answer some questions is actually down to feeling slightly intimidated by the onslaught of questions, their sterile aura, not knowing the person presenting them, etc... How much does the subconscious feeling of intimidation influence the ability to think?
  - topmonk 8y ago
    But that is just conjecture. No one knows why they failed the test more often, only they did. It could have something to do with their written language, for example, and shapes similar to triangles and squares in it appearing in a certain pattern that isn't the basic repeating XXOXXOXXO... Or it could be music and rhythm is a big part of their culture, and XXOXX then follows with XXXO in a large number of their songs.
mettamage 8y ago
I read the article. It barely scratches the surface of the issue, I was mad as a psychology student that this was the state of our knowledge when I read it in 2013 and didn't know about the publication crisis. The actual academic publication is really interesting and I'd recommend it as a read on whatever commute you're on. Here is the link: https://www.ssoar.info/ssoar/bitstream/handle/document/42104/ssoar-2010-henrich_et_al-The_weirdest_people_in_the.pdf?sequence=1 https://www.ssoar.info/ssoar/bitstream/handle/document/42104... Years after pondering this article, I felt I had figured it out: psychology is secretly the study of particular subset of 18 to 21 year old American women. The ones who study psychology because it's a female dominated study and all those psych students need their credit and 'participating' in research is part of it. Most of them are American because psychology is a bigger thing in the US than in Europe, or at least it seemed to be regarding well-known theories, so I presume most research happens there. There is another big group. A lot of dead mice (neuroscience).
- simple_phrases 8y ago
  > The ones who study psychology because it's a female dominated study and all those psych students need their credit and 'participating' in research is part of it. Wait, does participating in a study as a subject count for credit, or does working on a study count? Aren't studies explicit about the demographics of their participants? And don't studies that make generalized claims usually control for things like gender and age?
  - mettamage 8y ago
    I'm not sure whether this is a rant, or informing or both. Let me know if it's a bit too much on the ranting side, I'll edit it. > Wait, does participating in a study as a subject count for credit, or does working on a study count? There was this course called "measurements and diagnostics" and for that you had to pass the theory/tutorials of the course and also 10 hours of being a participant in research. If you didn't do the 10 hours of participation, you could do them at any moment you want and if you completed it, you immediately passed the course. My university has about 400 psychology students, so that is 4000 hours worth of people participating per year. And I'm pretty sure that more universities are doing this. In The Netherlands alone I estimate there to be about 40000 psychology students per year and probably about 5 hours of compulsory 'participation' in experimental studies. So a tiny country like The Netherlands is skewing this particular bias about 20000 participant hours per year. Participating in experimental studies is quite fun (pro tip: go for the fMRI and EEG studies ASAP! ;-) ), but I didn't appreciate how I was forced to do it. I didn't learn too much from those experiments and I felt used. It furthermore is a way to be lazy regarding participant recruitment. I'm pretty sure that some startup could disrupt this particular problem. If you have ideas about it, email me, I'm willing to brainstorm and help you to look for ways to solve this problem. > Aren't studies explicit about the demographics of their participants? And don't studies that make generalized claims usually control for things like gender and age? Studies are explicit about the demographics, which is why the WEIRD article could be written. And yes they do control for things like gender and age and in general they are quite careful making those claims. The issue is though as a research field psychology isn't the study of the human mind and human behavior but of WEIRD people their mind and WEIRD people their behavior. Like a lot of societal issues that are in popular media today, this is a systemic issue. Though I could imagine psychology researchers being like: "Hmm, I could get a representative group or I could just get some psychology students and churn my next paper out twice as fast!"
rando14775 8y ago
> [...] something like “I generally trust people.” Then participants are asked to choose one point along a five- or seven-point line ranging from strongly agree to strongly disagree. This numbered line is named a “Likert item” [...] Oh god, having filled out a bunch of these for diagnosis and such I hate these with a passion. I always wondered how well these actually work. I've seen grammatical nonsense like, "Do you often do X? -- always, often, sometimes, rarely, never". What, I often rarely do X? And what does often mean, anyway? Like once a week? Every day? Then, there are the abstract or vague questions that you then have to interpret what concrete situation it could apply to. Hard to think up an example off the top of my head, but how people reply to these surely depends on what exactly they think it might mean. Then you start losing patience after about 3 minutes of this shit, not to mention 15 or 30 minutes, and just go through them barely reading the questions, but for the first couple of questions you were pondering whether you "agree" or "somewhat agree" for ages.
- mettamage 8y ago
  Or what about people actively lying?
  - dvfjsdhgfv 8y ago
    I guess it would be a minority because telling the truth is easier and faster under most circumstances. But yes, all these studies whose the authors authoritatively make claims but these are are based on people answering questions regarding themselves (how often do you cheat on your partner?) - I never believe these numbers.
    - moonshinefe 8y ago
      Deception vs. honesty on certain questions in itself is a huge revealing factor in a person's personality.
    - selflesssieve 8y ago
      Not really - these Likert question studies are so much easier to click through rapidly. You can double or triple your hourly rate depending on how bold you are. Would a college student cheat their psychology department out of 10 dollars? Maybe some. I wonder how much shrinkage (wasted questionnaires) they have to account for? Pun semi-intended.
vbuwivbiu 8y ago
there could have been any number of blue triangles before that orange square - we see 2, but what about the rest out of frame
austincheney 8y ago
This is a known problem. Nearly all experimentation-based psychology research was in past decades conducted within the US and is heavily biased to US culture. Cultural differences aren't limited to how people interact with each other, but also include primitive qualities like how people perceive space and use tools.
yyyymmddhhmmss 8y ago
This isn’t even the worst problem with psychology. The worst is it’s contextual loyalties against critiquing the relationsips people have with one another. Power struggles, to be particular.
- apatters 8y ago
  Can you elaborate on this?
  - coldtea 8y ago
    "Let's not disturb the status quo and what's mainstream with our results, else we'll stop getting grants"
    - ativzzz 8y ago
      Isn't this applicable to most of academia and science and not just psychology?
      - coldtea 8y ago
        Physics and Engineering doesn't have many things that will annoy the status quo. While speaking against e.g. mainstream cultural tenets, or how we run society, or how we parent, etc., can lose you grants quickly, and those are all things psychology and sociology can get at.
  - yyyymmddhhmmss 8y ago
    By premeditating that diagnosis will be of the individual, the society is assumed a constant. In science, we call that fraud. Effectively, clinical psychology serves apology for societal and economic ills by blaming the effects on the individuals themselves. The process is strikingly transparent from an objective standpoint, but ironically it’s precisely an objective standpoint that is undermined by the institution. These days, you can visit one and tell them just about anything, and they will diagnose your disposition as illness. I don’t blame psychologists themselves, and who is to say they don’t help people? Many surely do, but the institution is an absolute drag on our economy and society; something like a collective excuse for avoiding self-critique. Sociology theoretically steps in here, but it is merely a study, and some branches of it would result in a totalitarian nightmare. What’s needed is humanist sociology and humanist psychology. As we learned in kindergarten class, there is nothing wrong with us. Each of us are unique and beautiful in our own ways. The questions are about how to live with one another. Pyschology as a study is another thing, but as a medical malpractice it has proven antithetical to systems design from the start.
est31 8y ago
The big problem with this is that the more diversity you have in your study participant set (gender, age, skin color, handedness, etc), the bigger your study has to be to correct for potential biases in results caused by differences in the participants unrelated to what you're testing. And the bigger the study, the more it'll cost, so there will be less studies overall.
- Cthulhu_ 8y ago
  I am glad that there are some big studies out there though; big in terms of participants, or big in terms of long-term research - about subjects like nutrition, cancer and heart problem risks, fertility, economic mobility, etc.
- SiempreViernes 8y ago
  It's not so much a problem as simply the cost of doing proper science, and just because you make a small study you don't have to make it cheaply (though you probably are if you're an academic).
- yakubin 8y ago
  I guess the dilemma is whether you want to do good science or just get your papers published for a low price.
- yoz-y 8y ago
  There should be incentive for laboratories to produce bigger and better studies rather than a bunch of small ones. Especially since a study with 300 participants is worth immensely more than 10 studies with 30 participants.
- taurath 8y ago
  Even among 18-21 year old college american women there are ridiculous amounts of variables that you can't control for. Give up the idea that you can control for biases in psychology. Do big mixed tests, then if there's positive outcomes see if its correlated to any of those groups. Then you can do more tests on the groups that didn't respond.
antwerpen 8y ago
I remember our psychology 101 professor offering .1 GPA “extra credit” for every psychology study you went to. There was no way to get a 4.0 in that class unless you attended every class, answered every exam question correctly, every quiz question, etc.
air7 8y ago
I actually don't see this is as too problematic. The entire field is murky because it's very hard to scientifically measure human psychology and behavior, and the tools of the trade seem almost laughably simplistic (like the aforementioned 5 point rating questionnaires). So our body of knowledge doesn't even reliably describe WEIRD people. Rather its a crude proxy, that just might contain some elements of truth warranting further investigation. But better crude tools than no tools, and better locally available subjects than no subjects (because most studies don't have the budget to go to Zambia). Similarly in other fields, research is done on rats pigs and monkeys with conclusions drawn to humans. Obviously not perfect, but again it's at best a starting point for later studies. I think the real problem is the over-zealous interpretation of study results as "truth".
- moonshinefe 8y ago
  I concur. Psychology isn't hard science by any mans, and extrapolating it to people outside your subjects' samples across the entire world into other cultures and things is obviously ridiculous. But we can still use tools and find patterns to help people. Surely, the solution isn't to throw up our hands in the West and be like "we can't find any help for the mentally ill because some subject might not apply to our discoveries from around here!"--but of course, if they are extrapolating those results to other cultures, that's worrisome and inaccurate in many studies. Human behavior has some universals, but a lot we figure are universal aren't.
- mjburgess 8y ago
  I disagree. If you're trying to measure a target 1cm across from 1 mile away with a ruler and a squint anything you say is not only likely wrong, but woefully deceptive. My view is that this shouldn't even be attempted because it just generates superstitious theories. Psychology is the skinner box pidegon. There's a fallacy here that's hard to pin down clearly but roughly: to measure inaccurately isnt to measure approximately. Its to measure totally in error. The errors in social psychology are not just "second decimal place", they're angels pushing stars.
  - slfnflctd 8y ago
hannob 8y ago
Reading the intro I had a strange feeling about this text. I don't have any issue with the general message. This surely is a legitimate problem. But the intro reads a bit like "psychology is such a great endeavour, if there wasn't this little issue." Everyone following science news should be aware by now that psychology suffers from a whole range of systemic methodological problems, notably publication bias, widespread p-hacking and failed replications.
alexstephen 8y ago
Interesting post shared by the author
alexstephen 8y ago
You can also get details https://www.5staressays.com/professional-essay-writers-for-hire https://www.5staressays.com/professional-essay-writers-for-h... by a team of professional writers as they have sound knowledge based on their research and development expertise
- nabla9 8y ago
  This is spam.
SubiculumCode 8y ago
Much of experimental design in psychology is an effort to attenuate the effect of individual differences so that you can find the common (and probably core) cognitive processes. Sometimes the risk to external validity is worth it in order to reduce the complexity of data and models from confounding variables. So we work with a system where most science is conducted with a relatively homogeneous sample of college undergraduates who are convenient to obtain, while well replicated or foundational findings are later tested across cultures to determine degree of generalization. It seems an ok compromise to me.
wisty 8y ago
That's not the only issue. https://en.wikipedia.org/wiki/Replication_crisis#Psychology_replication_rates https://en.wikipedia.org/wiki/Replication_crisis#Psychology_... > A report by the Open Science Collaboration in August 2015 that was coordinated by Brian Nosek estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.[38] Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies.
- JoeAltmaier 8y ago
  That makes perfect sense. The published studies are a sample of 'study-space' and that have outlier significance. Replicate published studies and their significance likely returns to the norm. Journals are filters to cherry-pick 'study space'. By the way they're constituted, they publish new studies that have overstated significance.
- 2019ideas 8y ago
  Causes of the replication crisis puts 0 blame on academia. This is an Academia caused crisis.
- Balgair 8y ago
  Veritasium has a good overview of the replication issues in much pf modern science: https://www.youtube.com/watch?v=42QuXLucH3Q https://www.youtube.com/watch?v=42QuXLucH3Q
xyk8jic0 8y ago
Reason number 92387429034809 for psychology to be considered pseudoscience.
ThomPete 8y ago
Psychology and social studies, antropology (and economy and whats called climate sciences) really shouldn't be called sciences. It's one of the rare cases IMO where the word really obstructs the core meaning of the subject matter. They should be considered temporary interpretations of statistical data or in short meta statistics cause that's really what they are. Much damage is being done by treating these fields as science and the article is only mentioning a few of those problems.
- dhimes 8y ago
  I think of it on a per-hypothesis or per-theory basis: If a hypothesis or theory is falsifiable, then it is scientific. If not, then it's not.
  - everdrive 8y ago
    No major disagreement with this general point of view. But, do you think that much of psychology has a falsifiability problem?
- danans 8y ago
  > They should be considered temporary interpretations of statistical data or in short meta statistics cause that's really what they are. This is what a lot of experimental sciences are, even physical ones, when the systems being studied are complex. There are very few areas of scientific study anymore that offer convenient, deterministic results. That fruit was picked a long time ago. Even at the cutting end of physics, researchers have to infer from statistical results. The difference is only that some of these fields have more reproducible results than others, often because they are studying less complex phenomena, whose causal factors and mechanisms can be more directly observed. Psychology is at one end of that spectrum, because it is studying the output of the mind, a biological information system whose mechanisms are among the most complex and obscure that people have ever studied.
  - ThomPete 8y ago
    I understand that and I agree with how you think about it but the problem is that it creates a very unhealthy public debate because one end of the spectrum gets confused with the other.
anonu 8y ago
75% of statistics are made up
nabla9 8y ago
The human behavioral and cultural adaptability to the environment is really impressive and it makes many psychological findings local. With technological and cultural changes some results may apply only to few generations before they go away. Think children growing in warzone or poor and violent environment. Their behaviour as adults is often sexually more promiscuous, aggressive and their impulse control seems to be less than 'the baseline'. They show trust issues. How much of that is just damage and disorder as psychology seems to assume, and how much is adaption to survive and procreate in an environment where lifespans are short and life is uncertain. Maybe childhood stress and stress hormones trigger survival strategies that work well in hard environment. They are maladaptive only in the culture and safety of the developed world.
mrcactu5 8y ago
a simple arithmetic computation shows that Europe has 750 million people, and the world has 8 billion people. That's 1/10 of the population. We neglect about 90% of people in... anything. South America has 500 million people, so they should get a comparable share, and Africa has 1.2 billion. So just by simple fractions we know something is off. A more careful study by subject and region could be required.
taurath 8y ago
Psychiatry is in the same boat. Anyone with multiple ongoing issues (depression + anxiety, for instance) is disqualified from studies. Guess what - those are the people that repeatedly have the worst outcomes or don't respond to treatment. We don't even know the real effect of smoking with ADHD people, because if they have anxiety they won't be included in the study.

245 points by headalgorithm 8y ago

Locus of control is another way of understanding how humans are not necessarily universal to one another by how events are perceived. The missing shape example might just be the children never encounter the square & triangles in daily life and are thrown off it. I don't really consider that a great example for showing how behavioural is different.

feanaro 8y ago
The very fact that familiarity with a shape can influence performance on pattern recognition tasks so dramatically is a significant finding on its own. It would mean we are effectively blind(er) to unfamiliar shapes, even though they are extremely simple, like a triangle or square are.
- bellerose 8y ago
  I’m surprised that is a recent discovery. I would think if someone asked a group of kids why someone would get the problem wrong. They would be like well first time seeing shapes.
  - feanaro 8y ago
    Well, some certainly do, but as always, it is a game of sampling, so individual errors are not as important as proportions. If we take the article at face value, proportions differ drastically between very different cultures so there must be something in the culture that biases the proportions towards one outcome or the other systematically. Of course, it's always possible any individual study contained unknown and unaccounted biases. Funding in science is still relatively small, the number of people participating few and science itself is hard.
- KoenDG 8y ago
  I was just thinking. If they gave me this test back in the '90s, but replaced it with a Pokeball and the face of a character from the Pokemon show, I'd probably have done better too. Those would have been things recognizable to me. Things I liked a lot. Also, they would have put my mind somewhat at ease, I think. I wonder how much of the inability to answer some questions is actually down to feeling slightly intimidated by the onslaught of questions, their sterile aura, not knowing the person presenting them, etc... How much does the subconscious feeling of intimidation influence the ability to think?
- topmonk 8y ago
  But that is just conjecture. No one knows why they failed the test more often, only they did. It could have something to do with their written language, for example, and shapes similar to triangles and squares in it appearing in a certain pattern that isn't the basic repeating XXOXXOXXO... Or it could be music and rhythm is a big part of their culture, and XXOXX then follows with XXXO in a large number of their songs.

mettamage 8y ago

I read the article. It barely scratches the surface of the issue, I was mad as a psychology student that this was the state of our knowledge when I read it in 2013 and didn't know about the publication crisis. The actual academic publication is really interesting and I'd recommend it as a read on whatever commute you're on. Here is the link: https://www.ssoar.info/ssoar/bitstream/handle/document/42104/ssoar-2010-henrich_et_al-The_weirdest_people_in_the.pdf?sequence=1 https://www.ssoar.info/ssoar/bitstream/handle/document/42104... Years after pondering this article, I felt I had figured it out: psychology is secretly the study of particular subset of 18 to 21 year old American women. The ones who study psychology because it's a female dominated study and all those psych students need their credit and 'participating' in research is part of it. Most of them are American because psychology is a bigger thing in the US than in Europe, or at least it seemed to be regarding well-known theories, so I presume most research happens there. There is another big group. A lot of dead mice (neuroscience).

simple_phrases 8y ago
> The ones who study psychology because it's a female dominated study and all those psych students need their credit and 'participating' in research is part of it. Wait, does participating in a study as a subject count for credit, or does working on a study count? Aren't studies explicit about the demographics of their participants? And don't studies that make generalized claims usually control for things like gender and age?
- mettamage 8y ago
  I'm not sure whether this is a rant, or informing or both. Let me know if it's a bit too much on the ranting side, I'll edit it. > Wait, does participating in a study as a subject count for credit, or does working on a study count? There was this course called "measurements and diagnostics" and for that you had to pass the theory/tutorials of the course and also 10 hours of being a participant in research. If you didn't do the 10 hours of participation, you could do them at any moment you want and if you completed it, you immediately passed the course. My university has about 400 psychology students, so that is 4000 hours worth of people participating per year. And I'm pretty sure that more universities are doing this. In The Netherlands alone I estimate there to be about 40000 psychology students per year and probably about 5 hours of compulsory 'participation' in experimental studies. So a tiny country like The Netherlands is skewing this particular bias about 20000 participant hours per year. Participating in experimental studies is quite fun (pro tip: go for the fMRI and EEG studies ASAP! ;-) ), but I didn't appreciate how I was forced to do it. I didn't learn too much from those experiments and I felt used. It furthermore is a way to be lazy regarding participant recruitment. I'm pretty sure that some startup could disrupt this particular problem. If you have ideas about it, email me, I'm willing to brainstorm and help you to look for ways to solve this problem. > Aren't studies explicit about the demographics of their participants? And don't studies that make generalized claims usually control for things like gender and age? Studies are explicit about the demographics, which is why the WEIRD article could be written. And yes they do control for things like gender and age and in general they are quite careful making those claims. The issue is though as a research field psychology isn't the study of the human mind and human behavior but of WEIRD people their mind and WEIRD people their behavior. Like a lot of societal issues that are in popular media today, this is a systemic issue. Though I could imagine psychology researchers being like: "Hmm, I could get a representative group or I could just get some psychology students and churn my next paper out twice as fast!"

rando14775 8y ago

> [...] something like “I generally trust people.” Then participants are asked to choose one point along a five- or seven-point line ranging from strongly agree to strongly disagree. This numbered line is named a “Likert item” [...] Oh god, having filled out a bunch of these for diagnosis and such I hate these with a passion. I always wondered how well these actually work. I've seen grammatical nonsense like, "Do you often do X? -- always, often, sometimes, rarely, never". What, I often rarely do X? And what does often mean, anyway? Like once a week? Every day? Then, there are the abstract or vague questions that you then have to interpret what concrete situation it could apply to. Hard to think up an example off the top of my head, but how people reply to these surely depends on what exactly they think it might mean. Then you start losing patience after about 3 minutes of this shit, not to mention 15 or 30 minutes, and just go through them barely reading the questions, but for the first couple of questions you were pondering whether you "agree" or "somewhat agree" for ages.

mettamage 8y ago
Or what about people actively lying?
- dvfjsdhgfv 8y ago
  I guess it would be a minority because telling the truth is easier and faster under most circumstances. But yes, all these studies whose the authors authoritatively make claims but these are are based on people answering questions regarding themselves (how often do you cheat on your partner?) - I never believe these numbers.
  - moonshinefe 8y ago
    Deception vs. honesty on certain questions in itself is a huge revealing factor in a person's personality.
  - selflesssieve 8y ago
    Not really - these Likert question studies are so much easier to click through rapidly. You can double or triple your hourly rate depending on how bold you are. Would a college student cheat their psychology department out of 10 dollars? Maybe some. I wonder how much shrinkage (wasted questionnaires) they have to account for? Pun semi-intended.

vbuwivbiu 8y ago

there could have been any number of blue triangles before that orange square - we see 2, but what about the rest out of frame

austincheney 8y ago

This is a known problem. Nearly all experimentation-based psychology research was in past decades conducted within the US and is heavily biased to US culture. Cultural differences aren't limited to how people interact with each other, but also include primitive qualities like how people perceive space and use tools.

yyyymmddhhmmss 8y ago

This isn’t even the worst problem with psychology. The worst is it’s contextual loyalties against critiquing the relationsips people have with one another. Power struggles, to be particular.

apatters 8y ago
Can you elaborate on this?
- coldtea 8y ago
  "Let's not disturb the status quo and what's mainstream with our results, else we'll stop getting grants"
  - ativzzz 8y ago
    Isn't this applicable to most of academia and science and not just psychology?
    - coldtea 8y ago
      Physics and Engineering doesn't have many things that will annoy the status quo. While speaking against e.g. mainstream cultural tenets, or how we run society, or how we parent, etc., can lose you grants quickly, and those are all things psychology and sociology can get at.
- yyyymmddhhmmss 8y ago
  By premeditating that diagnosis will be of the individual, the society is assumed a constant. In science, we call that fraud. Effectively, clinical psychology serves apology for societal and economic ills by blaming the effects on the individuals themselves. The process is strikingly transparent from an objective standpoint, but ironically it’s precisely an objective standpoint that is undermined by the institution. These days, you can visit one and tell them just about anything, and they will diagnose your disposition as illness. I don’t blame psychologists themselves, and who is to say they don’t help people? Many surely do, but the institution is an absolute drag on our economy and society; something like a collective excuse for avoiding self-critique. Sociology theoretically steps in here, but it is merely a study, and some branches of it would result in a totalitarian nightmare. What’s needed is humanist sociology and humanist psychology. As we learned in kindergarten class, there is nothing wrong with us. Each of us are unique and beautiful in our own ways. The questions are about how to live with one another. Pyschology as a study is another thing, but as a medical malpractice it has proven antithetical to systems design from the start.

est31 8y ago

The big problem with this is that the more diversity you have in your study participant set (gender, age, skin color, handedness, etc), the bigger your study has to be to correct for potential biases in results caused by differences in the participants unrelated to what you're testing. And the bigger the study, the more it'll cost, so there will be less studies overall.

Cthulhu_ 8y ago
I am glad that there are some big studies out there though; big in terms of participants, or big in terms of long-term research - about subjects like nutrition, cancer and heart problem risks, fertility, economic mobility, etc.
SiempreViernes 8y ago
It's not so much a problem as simply the cost of doing proper science, and just because you make a small study you don't have to make it cheaply (though you probably are if you're an academic).
yakubin 8y ago
I guess the dilemma is whether you want to do good science or just get your papers published for a low price.
yoz-y 8y ago
There should be incentive for laboratories to produce bigger and better studies rather than a bunch of small ones. Especially since a study with 300 participants is worth immensely more than 10 studies with 30 participants.
taurath 8y ago
Even among 18-21 year old college american women there are ridiculous amounts of variables that you can't control for. Give up the idea that you can control for biases in psychology. Do big mixed tests, then if there's positive outcomes see if its correlated to any of those groups. Then you can do more tests on the groups that didn't respond.

antwerpen 8y ago

I remember our psychology 101 professor offering .1 GPA “extra credit” for every psychology study you went to. There was no way to get a 4.0 in that class unless you attended every class, answered every exam question correctly, every quiz question, etc.

air7 8y ago

I actually don't see this is as too problematic. The entire field is murky because it's very hard to scientifically measure human psychology and behavior, and the tools of the trade seem almost laughably simplistic (like the aforementioned 5 point rating questionnaires). So our body of knowledge doesn't even reliably describe WEIRD people. Rather its a crude proxy, that just might contain some elements of truth warranting further investigation. But better crude tools than no tools, and better locally available subjects than no subjects (because most studies don't have the budget to go to Zambia). Similarly in other fields, research is done on rats pigs and monkeys with conclusions drawn to humans. Obviously not perfect, but again it's at best a starting point for later studies. I think the real problem is the over-zealous interpretation of study results as "truth".

moonshinefe 8y ago
I concur. Psychology isn't hard science by any mans, and extrapolating it to people outside your subjects' samples across the entire world into other cultures and things is obviously ridiculous. But we can still use tools and find patterns to help people. Surely, the solution isn't to throw up our hands in the West and be like "we can't find any help for the mentally ill because some subject might not apply to our discoveries from around here!"--but of course, if they are extrapolating those results to other cultures, that's worrisome and inaccurate in many studies. Human behavior has some universals, but a lot we figure are universal aren't.
mjburgess 8y ago
I disagree. If you're trying to measure a target 1cm across from 1 mile away with a ruler and a squint anything you say is not only likely wrong, but woefully deceptive. My view is that this shouldn't even be attempted because it just generates superstitious theories. Psychology is the skinner box pidegon. There's a fallacy here that's hard to pin down clearly but roughly: to measure inaccurately isnt to measure approximately. Its to measure totally in error. The errors in social psychology are not just "second decimal place", they're angels pushing stars.
- slfnflctd 8y ago

hannob 8y ago

Reading the intro I had a strange feeling about this text. I don't have any issue with the general message. This surely is a legitimate problem. But the intro reads a bit like "psychology is such a great endeavour, if there wasn't this little issue." Everyone following science news should be aware by now that psychology suffers from a whole range of systemic methodological problems, notably publication bias, widespread p-hacking and failed replications.

alexstephen 8y ago

Interesting post shared by the author

alexstephen 8y ago

You can also get details https://www.5staressays.com/professional-essay-writers-for-hire https://www.5staressays.com/professional-essay-writers-for-h... by a team of professional writers as they have sound knowledge based on their research and development expertise

nabla9 8y ago
This is spam.

SubiculumCode 8y ago

Much of experimental design in psychology is an effort to attenuate the effect of individual differences so that you can find the common (and probably core) cognitive processes. Sometimes the risk to external validity is worth it in order to reduce the complexity of data and models from confounding variables. So we work with a system where most science is conducted with a relatively homogeneous sample of college undergraduates who are convenient to obtain, while well replicated or foundational findings are later tested across cultures to determine degree of generalization. It seems an ok compromise to me.

wisty 8y ago

That's not the only issue. https://en.wikipedia.org/wiki/Replication_crisis#Psychology_replication_rates https://en.wikipedia.org/wiki/Replication_crisis#Psychology_... > A report by the Open Science Collaboration in August 2015 that was coordinated by Brian Nosek estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.[38] Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies.

JoeAltmaier 8y ago
That makes perfect sense. The published studies are a sample of 'study-space' and that have outlier significance. Replicate published studies and their significance likely returns to the norm. Journals are filters to cherry-pick 'study space'. By the way they're constituted, they publish new studies that have overstated significance.
2019ideas 8y ago
Causes of the replication crisis puts 0 blame on academia. This is an Academia caused crisis.
Balgair 8y ago
Veritasium has a good overview of the replication issues in much pf modern science: https://www.youtube.com/watch?v=42QuXLucH3Q https://www.youtube.com/watch?v=42QuXLucH3Q

xyk8jic0 8y ago

Reason number 92387429034809 for psychology to be considered pseudoscience.

ThomPete 8y ago

Psychology and social studies, antropology (and economy and whats called climate sciences) really shouldn't be called sciences. It's one of the rare cases IMO where the word really obstructs the core meaning of the subject matter. They should be considered temporary interpretations of statistical data or in short meta statistics cause that's really what they are. Much damage is being done by treating these fields as science and the article is only mentioning a few of those problems.

dhimes 8y ago
I think of it on a per-hypothesis or per-theory basis: If a hypothesis or theory is falsifiable, then it is scientific. If not, then it's not.
- everdrive 8y ago
  No major disagreement with this general point of view. But, do you think that much of psychology has a falsifiability problem?
danans 8y ago
> They should be considered temporary interpretations of statistical data or in short meta statistics cause that's really what they are. This is what a lot of experimental sciences are, even physical ones, when the systems being studied are complex. There are very few areas of scientific study anymore that offer convenient, deterministic results. That fruit was picked a long time ago. Even at the cutting end of physics, researchers have to infer from statistical results. The difference is only that some of these fields have more reproducible results than others, often because they are studying less complex phenomena, whose causal factors and mechanisms can be more directly observed. Psychology is at one end of that spectrum, because it is studying the output of the mind, a biological information system whose mechanisms are among the most complex and obscure that people have ever studied.
- ThomPete 8y ago
  I understand that and I agree with how you think about it but the problem is that it creates a very unhealthy public debate because one end of the spectrum gets confused with the other.

anonu 8y ago

75% of statistics are made up

nabla9 8y ago

The human behavioral and cultural adaptability to the environment is really impressive and it makes many psychological findings local. With technological and cultural changes some results may apply only to few generations before they go away. Think children growing in warzone or poor and violent environment. Their behaviour as adults is often sexually more promiscuous, aggressive and their impulse control seems to be less than 'the baseline'. They show trust issues. How much of that is just damage and disorder as psychology seems to assume, and how much is adaption to survive and procreate in an environment where lifespans are short and life is uncertain. Maybe childhood stress and stress hormones trigger survival strategies that work well in hard environment. They are maladaptive only in the culture and safety of the developed world.

mrcactu5 8y ago

a simple arithmetic computation shows that Europe has 750 million people, and the world has 8 billion people. That's 1/10 of the population. We neglect about 90% of people in... anything. South America has 500 million people, so they should get a comparable share, and Africa has 1.2 billion. So just by simple fractions we know something is off. A more careful study by subject and region could be required.

taurath 8y ago

Psychiatry is in the same boat. Anyone with multiple ongoing issues (depression + anxiety, for instance) is disqualified from studies. Guess what - those are the people that repeatedly have the worst outcomes or don't respond to treatment. We don't even know the real effect of smoking with ADHD people, because if they have anxiety they won't be included in the study.