5 ms·
This seems potentially interesting, but I am extremely skeptical that they can build a machine learning model that accurately diagnosis narcissistic or anxious
by PragmaticPulp 5y ago
This seems potentially interesting, but I am extremely skeptical that they can build a machine learning model that accurately diagnosis narcissistic or anxious behaviors from 280 character snippets and self-reported measures of narcissism:
> To collect our ground truth dataset for the prediction of narcissism, we recruited participants via an article published on the webpage of Psychology Today. At the end of the article, a link to a short survey was provided, in which, after providing consent, participants were asked to complete a scale on narcissism by Konrath et al. (2014), namely “To what extent do you agree with this statement: ‘I am a narcissist? (Note: The word ‘narcissist’ means egotistical, self-focused, and vain)” using a 7-point Likert scale (1 = “Not Very True of Me” to 7 = “Very True of Me”), and to provide their public Twitter username/handle for research purposes. In total, 1,067 participants, located in the United States, completed the survey, indicated their public Twitter username, and provided consent for their social media data to be used for research. In total, 309,417 tweets were obtained from these participants.
Psychology research already struggles with reproducibility issues at an alarmingly high rate. I fear that machining learning models are an easy way for researchers to get the answer they want from a dataset by tweaking the model until it produces the answer they want.
Their model included variables like average Tweet length and number of upper-case words. Notably, they built the model on one demographic (volunteer Psychology Today readers) but tested it on a demographic using Twitter for entirely different reasons (public-facing leaders of companies on Crunchbase)
It’s not hard to imagine a model like this creating a lot of false positives when applied to public-facing leaders of popular startups they found on Crunchbase
- dboreham 5y agoWondering how likely it is that a big-time narcissist will be reading Psychology Today.
- blululu 5y agoThere are plenty of highly narcissistic people with tenure and psychology departments are somewhat famous for being permeated with varying forms psychic peculiarities.
- themitigating 5y agoCan you provide proof of that?
- blululu 5y agoAbsolutely not. There are plenty of studies indicating that narcissist tend to get ahead in institutional contexts and I see no reason to think academia to be less viciously competitive than anywhere else. As for psych departments harboring a greater share of crazies, this is just a personal observation in an area that falls beyond the normally accepted range of research topics. Ask around, you might hear similar things; you might not.
- Cthulhu_ 5y agoThey will, looking for confirmation of how special they are.
- GavinMcG 5y agoSelf-reporting seems to be fairly accurate for narcissism: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0103469 https://journals.plos.org/plosone/article?id=10.1371/journal...
- PragmaticPulp 5y agoThey didn’t use the self-reported narcissists for the study, though. They were only used for training the model, which was then used to identify narcissist from a different set of Twitter “leaders” that they chose from Crunchbase: > Using an initial database of organizational leaders and employees, provided by Crunchbase (crunchbase.com), we randomly selected 500 highly active leaders on Twitter, and downloaded their public Twitter profile data. They trained the model on one demographic (readers of a psychology magazine) and then applied it to a different demographic (public faces of startups) and assumed that the model was equally accurate for this entirely different demographic.
- hirundo 5y ago> (Note: The word ‘narcissist’ means egotistical, self-focused, and vain)” using a 7-point Likert scale (1 = “Not Very True of Me” to 7 = “Very True of Me”) First, this study is clearly about me, but they don't admit that anywhere. Second, I am not a narcissist, because among all other people, I'm probably the least egotistical (I am but a modest demi-god), self-focused (the things I create are actually gifts to the world), or vain (my beauty is natural and effortless). So obviously I answered the poll with all ones, excluding myself from the training data and rendering this study pointless.
- austincheney 5y agoAnti-narcissism is generally challenging to identify as such suggests a gravity away from the self. A key identifier may be degrees of humility, however that’s defined, such that a person distrusts acknowledgements of success. Another identifier might be an inability to evaluate a group comparative to the self, which is the more rare high functioning side of Dunning-Kruger. Yet another identifier might be objective personality traits that preference balance and measures adversely to the self.
- ahthat 5y agoExactly. It’s easy to lie, narcissists more than others are likely to flatter themselves. This kind of data is not all that accurate or helpful. It really doesn’t tell you anything.
- Doxin 5y agoTurns out narcissists largely don't see narcissistic traits as a negative thing. Which makes just asking them about these sorts of things more effective than you'd imagine.