6 ms·
You're right. I had a look at the methods sections of three of the four studies cited in the article. It seems how they collect data is they have people fill in
by YeGoblynQueenne 7y ago
You're right. I had a look at the methods sections of three of the four
studies cited in the article. It seems how they collect data is they have
people fill in questionnaires, then calculate personality scores based on
those questionnaires.
For example, in the Mac Giolla and Kajonius study [1] the data came from an
online questionnaire.
I'm finding it very hard to criticise the methodology of this sort of study
without [edited out] being extremely rude and dismissive to a whole
field of research. But, I really don't understand how all this is supposed to
make sense. I look at the plot in the middle of the Scientific American
article that is marked "Agreeableness" on the x-axis (with a score from 1 to
5). I look at the papers cited. There's plenty of maths there, but what is
anything calculating? What is "agreeableness" and why is it measured on a
scale of 1 to 5? It seems to be a term that has a specific meaning in
psychology and sociology, and that ultimately translates to "a score of n in
this standardised questionnaire".
But, if you can just define whatever quantities you like and give them a
commonly used word for a name- then what does anything mean anymore? You can
just define anything you like as anything you like and measure it anyway you
like- and claim anything you want at all.
At the end of the day, is all this measuring anything other than trends in
filling up questionnaires? Can we really draw any other conclusions about the
differences of men and women than that the samples of the studies filled in
their questionnaires in different ways? Is even that a safe conclusion? If you
take statistics on a random process you can always model it- but you'll be
modelling noise. How is this possibility excluded here?
All the statistics are quantifying the answers that respondents gave to
questionnaires and how the researchers rated them - without any attempt to
blind or randomise anything, to protect from participant or researcher bias,
as far as I can tell (I'm searching for the word "blind" in the papers and
finding nothing). In the study I link below, participants found the website by
internet searches and word-of-mouth. The study itself points out that this is
a self-selecting sample, but what have they done to exclude the possibility of
adversarial participation (people purposefully filling in a form in a certain
way to confuse results)?
There is so much that is extremely precarious about the findings of those
studies and yet the Scientific American article jumps directly to d values.
Yes, but d-values on what? What is being measured? What do the numbers stand
for?
This is just heartbreaking to see that such a contentious issue is treated
with such frivolity. If it's not possible to lay to rest such hot button
issues with solid scientific work- then don't do it. It will just make matters
worse.
__________________
[1] https://onlinelibrary.wiley.com/doi/epdf/10.1002/ijop.12529 https://onlinelibrary.wiley.com/doi/epdf/10.1002/ijop.12529?
- datashow 7y ago> What is "agreeableness" Quote from the article https://www.frontiersin.org/articles/10.3389/fpsyg.2011.00178/full https://www.frontiersin.org/articles/10.3389/fpsyg.2011.0017...: "Agreeableness comprises traits relating to altruism, such as empathy and kindness. Agreeableness involves the tendency toward cooperation, maintenance of social harmony, and consideration of the concerns of others (as opposed to exploitation or victimization of others). Women consistently score higher than men on Agreeableness and related measures, such as tender-mindedness (Feingold, 1994; Costa et al., 2001)." > why is it measured on a scale of 1 to 5 Quote from the article https://www.frontiersin.org/articles/10.3389/fpsyg.2011.00178/full https://www.frontiersin.org/articles/10.3389/fpsyg.2011.0017...: "Participants rate their agreement with how well each statement describes them using a five-point scale ranging from strongly disagree to strongly agree." > But, if you can just define whatever quantities you like and give them a commonly used word for a name- then what does anything mean anymore? You can just define anything you like as anything you like and measure it anyway you like- and claim anything you want at all. > At the end of the day, is all this measuring anything other than trends in filling up questionnaires? Can we really draw any other conclusions about the differences of men and women than that the samples of the studies filled in their questionnaires in different ways? Is even that a safe conclusion? If you take statistics on a random process you can always model it- but you'll be modelling noise. How is this possibility excluded here? Not that simple. There is a whole field called Psychometrics. https://en.wikipedia.org/wiki/Psychometrics https://en.wikipedia.org/wiki/Psychometrics If you really care to answer those questions you need to take a complete psychometric theory course such as https://personality-project.org/revelle/syllabi/405.syllabus.html https://personality-project.org/revelle/syllabi/405.syllabus... Before you learned the whole thing, don't assume the field is as superficial as you imagined. > All the statistics are quantifying the answers that respondents gave to questionnaires and how the researchers rated them - without any attempt to blind or randomise anything, to protect from participant or researcher bias, as far as I can tell (I'm searching for the word "blind" in the papers and finding nothing). Your searching for the word "blind" means you don't know anything about psychology research. We don't use this word in our research. In psychology studies, we care about "reliability" and "validity" and we have extensive methods to test those. > In the study I link below, participants found the website by internet searches and word-of-mouth. The study itself points out that this is a self-selecting sample, but what have they done to exclude the possibility of adversarial participation (people purposefully filling in a form in a certain way to confuse results)? Maybe there are a few people try to do that. But with a sample size of 130,602, these response wouldn't impact the research findings at all, unless it is an organized effort trying to influence the research. > There is so much that is extremely precarious about the findings of those studies and yet the Scientific American article jumps directly to d values. Yes, but d-values on what? What is being measured? What do the numbers stand for? The ScientificAmerican article does fail to clear this up. But the first linked study using "D" clearly stated it is "Cattell's 16PF (fifth edition)". https://onlinelibrary.wiley.com/doi/pdf/10.1111/jopy.12500 https://onlinelibrary.wiley.com/doi/pdf/10.1111/jopy.12500 > This is just heartbreaking to see that such a contentious issue is treated with such frivolity. If it's not possible to lay to rest such hot button issues with solid scientific work- then don't do it. It will just make matters worse. Maybe the ScientificAmerican article is not flawless, but I think you need to calm down a little bit.