8 ms·

I think most of these are based on the Big Five, so they're defined from self reports. Personality questionnaires are far from ideal but they're what we've got

I think most of these are based on the Big Five, so they're defined from self reports. Personality questionnaires are far from ideal but they're what we've got and they're fairly reliable.

YeGoblynQueenne 7y ago
You're right. I had a look at the methods sections of three of the four studies cited in the article. It seems how they collect data is they have people fill in questionnaires, then calculate personality scores based on those questionnaires. For example, in the Mac Giolla and Kajonius study [1] the data came from an online questionnaire. I'm finding it very hard to criticise the methodology of this sort of study without [edited out] being extremely rude and dismissive to a whole field of research. But, I really don't understand how all this is supposed to make sense. I look at the plot in the middle of the Scientific American article that is marked "Agreeableness" on the x-axis (with a score from 1 to 5). I look at the papers cited. There's plenty of maths there, but what is anything calculating? What is "agreeableness" and why is it measured on a scale of 1 to 5? It seems to be a term that has a specific meaning in psychology and sociology, and that ultimately translates to "a score of n in this standardised questionnaire". But, if you can just define whatever quantities you like and give them a commonly used word for a name- then what does anything mean anymore? You can just define anything you like as anything you like and measure it anyway you like- and claim anything you want at all. At the end of the day, is all this measuring anything other than trends in filling up questionnaires? Can we really draw any other conclusions about the differences of men and women than that the samples of the studies filled in their questionnaires in different ways? Is even that a safe conclusion? If you take statistics on a random process you can always model it- but you'll be modelling noise. How is this possibility excluded here? All the statistics are quantifying the answers that respondents gave to questionnaires and how the researchers rated them - without any attempt to blind or randomise anything, to protect from participant or researcher bias, as far as I can tell (I'm searching for the word "blind" in the papers and finding nothing). In the study I link below, participants found the website by internet searches and word-of-mouth. The study itself points out that this is a self-selecting sample, but what have they done to exclude the possibility of adversarial participation (people purposefully filling in a form in a certain way to confuse results)? There is so much that is extremely precarious about the findings of those studies and yet the Scientific American article jumps directly to d values. Yes, but d-values on what? What is being measured? What do the numbers stand for? This is just heartbreaking to see that such a contentious issue is treated with such frivolity. If it's not possible to lay to rest such hot button issues with solid scientific work- then don't do it. It will just make matters worse. __________________ [1] https://onlinelibrary.wiley.com/doi/epdf/10.1002/ijop.12529 https://onlinelibrary.wiley.com/doi/epdf/10.1002/ijop.12529?
- datashow 7y ago
  > What is "agreeableness" Quote from the article https://www.frontiersin.org/articles/10.3389/fpsyg.2011.00178/full https://www.frontiersin.org/articles/10.3389/fpsyg.2011.0017...: "Agreeableness comprises traits relating to altruism, such as empathy and kindness. Agreeableness involves the tendency toward cooperation, maintenance of social harmony, and consideration of the concerns of others (as opposed to exploitation or victimization of others). Women consistently score higher than men on Agreeableness and related measures, such as tender-mindedness (Feingold, 1994; Costa et al., 2001)." > why is it measured on a scale of 1 to 5 Quote from the article https://www.frontiersin.org/articles/10.3389/fpsyg.2011.00178/full https://www.frontiersin.org/articles/10.3389/fpsyg.2011.0017...: "Participants rate their agreement with how well each statement describes them using a five-point scale ranging from strongly disagree to strongly agree." > But, if you can just define whatever quantities you like and give them a commonly used word for a name- then what does anything mean anymore? You can just define anything you like as anything you like and measure it anyway you like- and claim anything you want at all. > At the end of the day, is all this measuring anything other than trends in filling up questionnaires? Can we really draw any other conclusions about the differences of men and women than that the samples of the studies filled in their questionnaires in different ways? Is even that a safe conclusion? If you take statistics on a random process you can always model it- but you'll be modelling noise. How is this possibility excluded here? Not that simple. There is a whole field called Psychometrics. https://en.wikipedia.org/wiki/Psychometrics https://en.wikipedia.org/wiki/Psychometrics If you really care to answer those questions you need to take a complete psychometric theory course such as https://personality-project.org/revelle/syllabi/405.syllabus.html https://personality-project.org/revelle/syllabi/405.syllabus... Before you learned the whole thing, don't assume the field is as superficial as you imagined. > All the statistics are quantifying the answers that respondents gave to questionnaires and how the researchers rated them - without any attempt to blind or randomise anything, to protect from participant or researcher bias, as far as I can tell (I'm searching for the word "blind" in the papers and finding nothing). Your searching for the word "blind" means you don't know anything about psychology research. We don't use this word in our research. In psychology studies, we care about "reliability" and "validity" and we have extensive methods to test those. > In the study I link below, participants found the website by internet searches and word-of-mouth. The study itself points out that this is a self-selecting sample, but what have they done to exclude the possibility of adversarial participation (people purposefully filling in a form in a certain way to confuse results)? Maybe there are a few people try to do that. But with a sample size of 130,602, these response wouldn't impact the research findings at all, unless it is an organized effort trying to influence the research. > There is so much that is extremely precarious about the findings of those studies and yet the Scientific American article jumps directly to d values. Yes, but d-values on what? What is being measured? What do the numbers stand for? The ScientificAmerican article does fail to clear this up. But the first linked study using "D" clearly stated it is "Cattell's 16PF (fifth edition)". https://onlinelibrary.wiley.com/doi/pdf/10.1111/jopy.12500 https://onlinelibrary.wiley.com/doi/pdf/10.1111/jopy.12500 > This is just heartbreaking to see that such a contentious issue is treated with such frivolity. If it's not possible to lay to rest such hot button issues with solid scientific work- then don't do it. It will just make matters worse. Maybe the ScientificAmerican article is not flawless, but I think you need to calm down a little bit.

I think most of these are based on the Big Five, so they're defined from self reports. Personality questionnaires are far from ideal but they're what we've got and they're fairly reliable.

YeGoblynQueenne 7y ago

You're right. I had a look at the methods sections of three of the four studies cited in the article. It seems how they collect data is they have people fill in questionnaires, then calculate personality scores based on those questionnaires. For example, in the Mac Giolla and Kajonius study [1] the data came from an online questionnaire. I'm finding it very hard to criticise the methodology of this sort of study without [edited out] being extremely rude and dismissive to a whole field of research. But, I really don't understand how all this is supposed to make sense. I look at the plot in the middle of the Scientific American article that is marked "Agreeableness" on the x-axis (with a score from 1 to 5). I look at the papers cited. There's plenty of maths there, but what is anything calculating? What is "agreeableness" and why is it measured on a scale of 1 to 5? It seems to be a term that has a specific meaning in psychology and sociology, and that ultimately translates to "a score of n in this standardised questionnaire". But, if you can just define whatever quantities you like and give them a commonly used word for a name- then what does anything mean anymore? You can just define anything you like as anything you like and measure it anyway you like- and claim anything you want at all. At the end of the day, is all this measuring anything other than trends in filling up questionnaires? Can we really draw any other conclusions about the differences of men and women than that the samples of the studies filled in their questionnaires in different ways? Is even that a safe conclusion? If you take statistics on a random process you can always model it- but you'll be modelling noise. How is this possibility excluded here? All the statistics are quantifying the answers that respondents gave to questionnaires and how the researchers rated them - without any attempt to blind or randomise anything, to protect from participant or researcher bias, as far as I can tell (I'm searching for the word "blind" in the papers and finding nothing). In the study I link below, participants found the website by internet searches and word-of-mouth. The study itself points out that this is a self-selecting sample, but what have they done to exclude the possibility of adversarial participation (people purposefully filling in a form in a certain way to confuse results)? There is so much that is extremely precarious about the findings of those studies and yet the Scientific American article jumps directly to d values. Yes, but d-values on what? What is being measured? What do the numbers stand for? This is just heartbreaking to see that such a contentious issue is treated with such frivolity. If it's not possible to lay to rest such hot button issues with solid scientific work- then don't do it. It will just make matters worse. __________________ [1] https://onlinelibrary.wiley.com/doi/epdf/10.1002/ijop.12529 https://onlinelibrary.wiley.com/doi/epdf/10.1002/ijop.12529?

datashow 7y ago
> What is "agreeableness" Quote from the article https://www.frontiersin.org/articles/10.3389/fpsyg.2011.00178/full https://www.frontiersin.org/articles/10.3389/fpsyg.2011.0017...: "Agreeableness comprises traits relating to altruism, such as empathy and kindness. Agreeableness involves the tendency toward cooperation, maintenance of social harmony, and consideration of the concerns of others (as opposed to exploitation or victimization of others). Women consistently score higher than men on Agreeableness and related measures, such as tender-mindedness (Feingold, 1994; Costa et al., 2001)." > why is it measured on a scale of 1 to 5 Quote from the article https://www.frontiersin.org/articles/10.3389/fpsyg.2011.00178/full https://www.frontiersin.org/articles/10.3389/fpsyg.2011.0017...: "Participants rate their agreement with how well each statement describes them using a five-point scale ranging from strongly disagree to strongly agree." > But, if you can just define whatever quantities you like and give them a commonly used word for a name- then what does anything mean anymore? You can just define anything you like as anything you like and measure it anyway you like- and claim anything you want at all. > At the end of the day, is all this measuring anything other than trends in filling up questionnaires? Can we really draw any other conclusions about the differences of men and women than that the samples of the studies filled in their questionnaires in different ways? Is even that a safe conclusion? If you take statistics on a random process you can always model it- but you'll be modelling noise. How is this possibility excluded here? Not that simple. There is a whole field called Psychometrics. https://en.wikipedia.org/wiki/Psychometrics https://en.wikipedia.org/wiki/Psychometrics If you really care to answer those questions you need to take a complete psychometric theory course such as https://personality-project.org/revelle/syllabi/405.syllabus.html https://personality-project.org/revelle/syllabi/405.syllabus... Before you learned the whole thing, don't assume the field is as superficial as you imagined. > All the statistics are quantifying the answers that respondents gave to questionnaires and how the researchers rated them - without any attempt to blind or randomise anything, to protect from participant or researcher bias, as far as I can tell (I'm searching for the word "blind" in the papers and finding nothing). Your searching for the word "blind" means you don't know anything about psychology research. We don't use this word in our research. In psychology studies, we care about "reliability" and "validity" and we have extensive methods to test those. > In the study I link below, participants found the website by internet searches and word-of-mouth. The study itself points out that this is a self-selecting sample, but what have they done to exclude the possibility of adversarial participation (people purposefully filling in a form in a certain way to confuse results)? Maybe there are a few people try to do that. But with a sample size of 130,602, these response wouldn't impact the research findings at all, unless it is an organized effort trying to influence the research. > There is so much that is extremely precarious about the findings of those studies and yet the Scientific American article jumps directly to d values. Yes, but d-values on what? What is being measured? What do the numbers stand for? The ScientificAmerican article does fail to clear this up. But the first linked study using "D" clearly stated it is "Cattell's 16PF (fifth edition)". https://onlinelibrary.wiley.com/doi/pdf/10.1111/jopy.12500 https://onlinelibrary.wiley.com/doi/pdf/10.1111/jopy.12500 > This is just heartbreaking to see that such a contentious issue is treated with such frivolity. If it's not possible to lay to rest such hot button issues with solid scientific work- then don't do it. It will just make matters worse. Maybe the ScientificAmerican article is not flawless, but I think you need to calm down a little bit.