4 ms·

Can't read the full article for free, but I expect their measurement techniques are tautological. It's the reification fallacy -- create a measurement, redefine

Can't read the full article for free, but I expect their measurement techniques are tautological. It's the reification fallacy -- create a measurement, redefine the term as that measurement, then conflate the measurement with the original meaning of the term.

imglorp 11y ago
Not an experimenter here, but I can't imagine using Mechanical Turk will provide any useful data whatsoever. The only thing MT does is train people to click its buttons as fast as possible to move on to the next task. If someone has figured out the captcha, there might not even be a human making inputs.
closed 11y ago
To be sure, there are two steps in their process: convince you their assessment is a valid reflection of the construct (term) of interest, and then examine different properties of the assessment (e.g. how reliable are a person's responses to this assessment). The convince you part is why it is not necessarily the reification fallacy. For example, suppose I create a measure of sadness that is the sole question: are you sad?. Does it seem like a good measure of the term sadness to you? If you think it's a bad measure,then I haven't convinced you, and we need to rethink a more convincing assessment, but it doesn't mean that if I use that question to operationalize (serve as my behavioral definition for) sadness that I committed the reification fallacy. Instead, you've failed to recognize that I'm using an argument of the form, "if you believe this is a good behavioral definition of sadness, then...". If you could ever see yourself using a statment like "that person is sad", then there are probably observables I could use to do research based on what sadness means to you. That's basically the logic these and many researchers in psychology / neuroscience are taking.
fancy_pantser 11y ago
Full article is available online. They used a series of surveys to correlate personality traits to people that needed or created drama in the eyes of the surveyed. They then create a composite measure of the strongest signals amongst the traits. http://scottbarrykaufman.com/wp-content/uploads/2016/01/Frankowski-et-al.-2016.pdf http://scottbarrykaufman.com/wp-content/uploads/2016/01/Fran...

imglorp 11y ago

Not an experimenter here, but I can't imagine using Mechanical Turk will provide any useful data whatsoever. The only thing MT does is train people to click its buttons as fast as possible to move on to the next task. If someone has figured out the captcha, there might not even be a human making inputs.

closed 11y ago

To be sure, there are two steps in their process: convince you their assessment is a valid reflection of the construct (term) of interest, and then examine different properties of the assessment (e.g. how reliable are a person's responses to this assessment). The convince you part is why it is not necessarily the reification fallacy. For example, suppose I create a measure of sadness that is the sole question: are you sad?. Does it seem like a good measure of the term sadness to you? If you think it's a bad measure,then I haven't convinced you, and we need to rethink a more convincing assessment, but it doesn't mean that if I use that question to operationalize (serve as my behavioral definition for) sadness that I committed the reification fallacy. Instead, you've failed to recognize that I'm using an argument of the form, "if you believe this is a good behavioral definition of sadness, then...". If you could ever see yourself using a statment like "that person is sad", then there are probably observables I could use to do research based on what sadness means to you. That's basically the logic these and many researchers in psychology / neuroscience are taking.

fancy_pantser 11y ago

Full article is available online. They used a series of surveys to correlate personality traits to people that needed or created drama in the eyes of the surveyed. They then create a composite measure of the strongest signals amongst the traits. http://scottbarrykaufman.com/wp-content/uploads/2016/01/Frankowski-et-al.-2016.pdf http://scottbarrykaufman.com/wp-content/uploads/2016/01/Fran...