4 ms·

Did they really not control for race? I didn't close read but I can't fathom not controlling for race/nationality, which I'd suspect is correlated with need for

Did they really not control for race? I didn't close read but I can't fathom not controlling for race/nationality, which I'd suspect is correlated with need for uniqueness.

tgv 2y ago
It's an online survey. This data is as unrepresentative and unreliable as they come, and the effects are small. This study is one to ignore.
- alwa 2y ago
  With 1.3 million samples showing fairly clean/consistent trends, something seems to be reliably happening, sustained over 20 years. And effect sizes on the order of 1/3 of a standard deviation seem not that small: an average score in 2000 would be somewhere around the 60th percentile if you got it today, right? They did include demographic questions between 2004 and 2009, which seemed to line up with other results out of that lab [0] indicating that massive online panels like this are reasonably representative, if skewed female. As nice as it would have been if they’d looked more formally, there seems not to be much indication that the demographic composition of their respondents changed all that much over that time. Even if these results only describe “the population of online people who do personality tests for fun,” don’t they still reflect something reliably changing about that population? And whatever we wish had been done differently in this study, isn’t it appropriate to interpret it as at least a flawed piece of evidence against the entire landscape of clues pointing toward a rise in social anxiety over that time? [0] Sam Gosling, whose project collected these data, wrote the book about massive online panels: https://www.apa.org/pubs/books/4311014 https://www.apa.org/pubs/books/4311014 . The project this data came from started off with research addressing whether it was representative enough to be useful https://psycnet.apa.org/doiLanding?doi=10.1037/0003-066X.59.2.93 https://psycnet.apa.org/doiLanding?doi=10.1037/0003-066X.59.... . Lord knows lessons have been learned since then, but I’m not sure it’s fair to imply that serious people didn’t think seriously about the questions we’re raising here.
  - tgv 2y ago
    > something That's the keyword. Something, not "need for uniqueness." The change can be anything, e.g. a change in the respondent population, or a trend from desktop to mobile (which has been reported to have an effect on how people answer scale questions). E.g., if you look at the graphs, there's an increase in standard deviation over time, and a pretty big one too. Where does that come from? You can't just say "hey, I have a post-hoc hypothesis, some very noisy data, and some correlation fits, so I must be right." > isn’t it appropriate to interpret it as at least a flawed piece of evidence against ... a rise in social anxiety In what way? First, need for uniqueness and social anxiety are two different things. Second, the need for uniqueness may have risen, while other trends are pushing the line downwards. And third, the relation between the questionnaire and need for uniqueness is tentative, to say the least. One of the questions they included is “It is better to always agree with the opinion of others than to be considered a disagreeable person.” It's all hand waving. So no, I can't agree, and it would be unethical to draw any real-life conclusions from such an article. > I’m not sure it’s fair to imply that serious people didn’t think seriously It is naive to think they found a solution for that, and forgot to mention it in the article. The fact that demographic data wasn't available for 3/4 of the respondents should give that thought a pause. Academics don't work for the truth, but for publications and citations. It's the main reason so many studies in psychology are simply false.

Did they really not control for race? I didn't close read but I can't fathom not controlling for race/nationality, which I'd suspect is correlated with need for uniqueness.

tgv 2y ago

It's an online survey. This data is as unrepresentative and unreliable as they come, and the effects are small. This study is one to ignore.

alwa 2y ago
With 1.3 million samples showing fairly clean/consistent trends, something seems to be reliably happening, sustained over 20 years. And effect sizes on the order of 1/3 of a standard deviation seem not that small: an average score in 2000 would be somewhere around the 60th percentile if you got it today, right? They did include demographic questions between 2004 and 2009, which seemed to line up with other results out of that lab [0] indicating that massive online panels like this are reasonably representative, if skewed female. As nice as it would have been if they’d looked more formally, there seems not to be much indication that the demographic composition of their respondents changed all that much over that time. Even if these results only describe “the population of online people who do personality tests for fun,” don’t they still reflect something reliably changing about that population? And whatever we wish had been done differently in this study, isn’t it appropriate to interpret it as at least a flawed piece of evidence against the entire landscape of clues pointing toward a rise in social anxiety over that time? [0] Sam Gosling, whose project collected these data, wrote the book about massive online panels: https://www.apa.org/pubs/books/4311014 https://www.apa.org/pubs/books/4311014 . The project this data came from started off with research addressing whether it was representative enough to be useful https://psycnet.apa.org/doiLanding?doi=10.1037/0003-066X.59.2.93 https://psycnet.apa.org/doiLanding?doi=10.1037/0003-066X.59.... . Lord knows lessons have been learned since then, but I’m not sure it’s fair to imply that serious people didn’t think seriously about the questions we’re raising here.
- tgv 2y ago
  > something That's the keyword. Something, not "need for uniqueness." The change can be anything, e.g. a change in the respondent population, or a trend from desktop to mobile (which has been reported to have an effect on how people answer scale questions). E.g., if you look at the graphs, there's an increase in standard deviation over time, and a pretty big one too. Where does that come from? You can't just say "hey, I have a post-hoc hypothesis, some very noisy data, and some correlation fits, so I must be right." > isn’t it appropriate to interpret it as at least a flawed piece of evidence against ... a rise in social anxiety In what way? First, need for uniqueness and social anxiety are two different things. Second, the need for uniqueness may have risen, while other trends are pushing the line downwards. And third, the relation between the questionnaire and need for uniqueness is tentative, to say the least. One of the questions they included is “It is better to always agree with the opinion of others than to be considered a disagreeable person.” It's all hand waving. So no, I can't agree, and it would be unethical to draw any real-life conclusions from such an article. > I’m not sure it’s fair to imply that serious people didn’t think seriously It is naive to think they found a solution for that, and forgot to mention it in the article. The fact that demographic data wasn't available for 3/4 of the respondents should give that thought a pause. Academics don't work for the truth, but for publications and citations. It's the main reason so many studies in psychology are simply false.