4 ms·

I'm always amazed that someone who by all rights should be intimately familiar with the history of social science research like McIntyre can seemingly be so una

I'm always amazed that someone who by all rights should be intimately familiar with the history of social science research like McIntyre can seemingly be so unaware of it. There was an empirical, modernist period in social sciences in the 40-80s of exactly the kind suggested by Lee and it was an absolute disaster. The results couldn't be related to the real world, and no one was really able to get over the inherent [1] influence of the author in the data. After a lot of arguing and furious paper writing in the late 80s-90s, everyone began working on how to deal with that bias, moving it front and center where it can be handled instead of subtly hiding it in a bunch of numbers and pretending it didn't exist. Does it make people who aren't familiar with what's going on think everyone is a bunch of raving lunatics? Yes. It creates a heck of an image problem, not helped by the fact that some smaller percentage of academics are ideological lunatics. But it's far more manageable for their peers to deal with than trying to solve an impossible problem. [1] If you don't understand why that's an inherent problem, ask yourself whether a white male like myself would get the same results doing research on menstruation in Kabul as an Afghani woman would, or an Arab doing sentiment polls immediately after 9/11.

gridlockd 7y ago
> After a lot of arguing and furious paper writing in the late 80s-90s, everyone began working on how to deal with that bias, moving it front and center where it can be handled instead of subtly hiding it in a bunch of numbers and pretending it didn't exist. And the result of that is... what exactly?
- thrwayxyz 7y ago
  The homeless epidemic mostly. Sometimes doing badly is better than doing terribly. In the name of the perfect and being 'non prejudiced' the current generation of psychologists have doomed the mentally ill to living standards lower than the 19th century and a life expectancy on par.
  - mancerayder 7y ago
    That's a bold claim, perhaps worthy of elaboration.
- AlotOfReading 7y ago
  Vastly improved methodology and understanding of results. One of the issues of modernist research was that objective data was trash, but no one knew it. For instance, anthropologists would throw out / fail to collect artifacts that weren't usable with the analyses available at the time. You can't recover that from a table that says 50 bone shards were found in a pit. You can begin to recover that if the researcher flat out says they don't believe small bones are useful for the easily (an ideological position). Or, to use the example I gave earlier, if both a white male researcher and a female Afghani researcher studied menstruation in Kabul, they very well might get different results without any deficiencies in methodology by either. You can start to explain that if the researchers both flat out state their approaches and assumptions in interacting with their female subjects. Doing that requires us to understand that the researcher affects the results though, which is a fundamentally different view than standard empiricism.
  - gridlockd 7y ago
    > You can start to explain that if the researchers both flat out state their approaches and assumptions in interacting with their female subjects. I would think that the foundation behind your argument - which I agree with - implies that this isn't actually possible. You have an infinite regress of biases. Empirically, what you can do - to take your example of those menstruation researchers - is to get more data from researchers of different backgrounds to try and statistically isolate those biases. This is part of "standard empiricism". You also always have to live with the fact that data is sometimes too noisy to be meaningful.
choeger 7y ago
> The results couldn't be related to the real world, and no one was really able to get over the inherent [1] influence of the author in the data. Wait. Do I understand you correctly that an objective approach failed and your conclusion is to be more subjective?
- AlotOfReading 7y ago
  The "objective" approach was never actually objective. It had just as many biases and issues as everything after it, but everyone sort of pretended it didn't. The change was recognizing that the same data wasn't objective and that researchers couldn't make it so. Instead, theory shifted to analyzing those inherent biases and studying how they affected research.
  - malvosenior 7y ago
    If your data is not good enough, get better data. Don't abandon objectivity. I think this is why a lot of people don't take the social sciences seriously, there's an extreme resistance to just doing objective, data driven research (and the lack of reproducibility that comes with that).
    - jftj 7y ago
      It's not that "the data is not good enough". It's that it's easy to obfuscate results by hiding bias in the data, even if unintentionally. Even in the "hard" sciences. Or have we forgotten about some of the awful research being done in machine learning? Remember the one about identifying future criminals from photos?
    - ajb 7y ago
      It took 200 years for the hard sciences to figure out the right way to 'do' objectivity, and arguably it's still evolving (see 'Objectivity'[1] by Daston and Galison, nice summary here: https://scienceobjectivity.weebly.com/daston--galison.html https://scienceobjectivity.weebly.com/daston--galison.html). The social sciences have additional problems, saying 'just copy the hard sciences' isn't reasonable. In fact, the idea that objectivity can be done in a simple mechanical way doesn't work for the hard sciences either, and was rejected by them in the early 20th, because it turns out that interpreting the evidence objectively still requires an educated eye. If you've ever had the good fortune (or misfortune, considering the circs) to see one of your own x-rays, you'll know how much you have to take the word of the radiographer for what the photo actually means. Sure, eventually, after much debate, a mechanical formula for interpreting the photo may be developed. Radiographers are probably due to be replaced by AI any day now - but their judgements will have formed the labels of the training data. At the cutting edge of the sciences, you don't have any of that yet. [1] that book was expanded from their earlier paper which can be found here: http://cspeech.ucd.ie/Fred/docs/Galison.pdf http://cspeech.ucd.ie/Fred/docs/Galison.pdf
nradov 7y ago
This is an intellectually lazy comment. It is by no means obvious that the gender and ethnicity of of the researcher would impact that hypothetical experiment, or what the impact would be, or that it couldn't be avoided with careful experimental design. If the social sciences had a better general track record of producing reproducible results with practical value for society then I would be inclined to give them the benefit of the doubt. But the reality is that most researchers have squandered their credibility by chasing mirages. If it were up to me I would cut funding for the entire field.
- akiselev 7y ago
  > If the social sciences had a better general track record of producing reproducible results with practical value for society then I would be inclined to give them the benefit of the doubt. But the reality is that most researchers have squandered their credibility by chasing mirages. If it were up to me I would cut funding for the entire field. Is that not the definition of intellectual laziness? You've just given up intellectually on an entire field.
- FranzFerdiNaN 7y ago
  Why not go one step further and only fund math, because in the end everything is math, right? No need to fund physics, which is just applied math. No need to fund medicine, which is just applied math. And medicine is a field filled with terrible studes that cant be replicated, so clearly it cant be good for anything.
abdullahkhalids 7y ago
> After a lot of arguing and furious paper writing in the late 80s-90s, Can you cite at least one paper from this series?
- AlotOfReading 7y ago
  My background is in archaeology, so that's what I'll discuss. The other social sciences had similar debates around the same time, plus or minus a few years. If you're looking to understand one perspective from this period, I'd actually recommend Binford's Debating Archaeology [1]. This book summarizes a whole slew of arguments Binford had in the 80s with a number of other archaeologists, including his famous spate with Gould over the limits of empiricism. As it's a summary, it was obviously written after all of this was settled. Selected works from the era might be Hodder's Symbolic and Structural Archaeology[2] or kohl's "Limits to Post-Processual Archaeology" [3]. The former was essentially an opening salvo for this debate and the latter is an example of a the reactions to it. [1] Binford, L. R. (2016). Debating Archaeology: Updated Edition. Routledge. [2] Hodder, I. (1982). Symbolic and Structural Archaeology. [3] Kohl, P. L. (1993). Limits to a post-processual archaeology (or, The dangers of a new scholasticism). Archaeological Theory: who sets the agenda, 13-19.

gridlockd 7y ago

> After a lot of arguing and furious paper writing in the late 80s-90s, everyone began working on how to deal with that bias, moving it front and center where it can be handled instead of subtly hiding it in a bunch of numbers and pretending it didn't exist. And the result of that is... what exactly?

thrwayxyz 7y ago
The homeless epidemic mostly. Sometimes doing badly is better than doing terribly. In the name of the perfect and being 'non prejudiced' the current generation of psychologists have doomed the mentally ill to living standards lower than the 19th century and a life expectancy on par.
- mancerayder 7y ago
  That's a bold claim, perhaps worthy of elaboration.
AlotOfReading 7y ago
Vastly improved methodology and understanding of results. One of the issues of modernist research was that objective data was trash, but no one knew it. For instance, anthropologists would throw out / fail to collect artifacts that weren't usable with the analyses available at the time. You can't recover that from a table that says 50 bone shards were found in a pit. You can begin to recover that if the researcher flat out says they don't believe small bones are useful for the easily (an ideological position). Or, to use the example I gave earlier, if both a white male researcher and a female Afghani researcher studied menstruation in Kabul, they very well might get different results without any deficiencies in methodology by either. You can start to explain that if the researchers both flat out state their approaches and assumptions in interacting with their female subjects. Doing that requires us to understand that the researcher affects the results though, which is a fundamentally different view than standard empiricism.
- gridlockd 7y ago
  > You can start to explain that if the researchers both flat out state their approaches and assumptions in interacting with their female subjects. I would think that the foundation behind your argument - which I agree with - implies that this isn't actually possible. You have an infinite regress of biases. Empirically, what you can do - to take your example of those menstruation researchers - is to get more data from researchers of different backgrounds to try and statistically isolate those biases. This is part of "standard empiricism". You also always have to live with the fact that data is sometimes too noisy to be meaningful.

choeger 7y ago

> The results couldn't be related to the real world, and no one was really able to get over the inherent [1] influence of the author in the data. Wait. Do I understand you correctly that an objective approach failed and your conclusion is to be more subjective?

AlotOfReading 7y ago
The "objective" approach was never actually objective. It had just as many biases and issues as everything after it, but everyone sort of pretended it didn't. The change was recognizing that the same data wasn't objective and that researchers couldn't make it so. Instead, theory shifted to analyzing those inherent biases and studying how they affected research.
- malvosenior 7y ago
  If your data is not good enough, get better data. Don't abandon objectivity. I think this is why a lot of people don't take the social sciences seriously, there's an extreme resistance to just doing objective, data driven research (and the lack of reproducibility that comes with that).
  - jftj 7y ago
    It's not that "the data is not good enough". It's that it's easy to obfuscate results by hiding bias in the data, even if unintentionally. Even in the "hard" sciences. Or have we forgotten about some of the awful research being done in machine learning? Remember the one about identifying future criminals from photos?
  - ajb 7y ago
    It took 200 years for the hard sciences to figure out the right way to 'do' objectivity, and arguably it's still evolving (see 'Objectivity'[1] by Daston and Galison, nice summary here: https://scienceobjectivity.weebly.com/daston--galison.html https://scienceobjectivity.weebly.com/daston--galison.html). The social sciences have additional problems, saying 'just copy the hard sciences' isn't reasonable. In fact, the idea that objectivity can be done in a simple mechanical way doesn't work for the hard sciences either, and was rejected by them in the early 20th, because it turns out that interpreting the evidence objectively still requires an educated eye. If you've ever had the good fortune (or misfortune, considering the circs) to see one of your own x-rays, you'll know how much you have to take the word of the radiographer for what the photo actually means. Sure, eventually, after much debate, a mechanical formula for interpreting the photo may be developed. Radiographers are probably due to be replaced by AI any day now - but their judgements will have formed the labels of the training data. At the cutting edge of the sciences, you don't have any of that yet. [1] that book was expanded from their earlier paper which can be found here: http://cspeech.ucd.ie/Fred/docs/Galison.pdf http://cspeech.ucd.ie/Fred/docs/Galison.pdf

nradov 7y ago

This is an intellectually lazy comment. It is by no means obvious that the gender and ethnicity of of the researcher would impact that hypothetical experiment, or what the impact would be, or that it couldn't be avoided with careful experimental design. If the social sciences had a better general track record of producing reproducible results with practical value for society then I would be inclined to give them the benefit of the doubt. But the reality is that most researchers have squandered their credibility by chasing mirages. If it were up to me I would cut funding for the entire field.

akiselev 7y ago
> If the social sciences had a better general track record of producing reproducible results with practical value for society then I would be inclined to give them the benefit of the doubt. But the reality is that most researchers have squandered their credibility by chasing mirages. If it were up to me I would cut funding for the entire field. Is that not the definition of intellectual laziness? You've just given up intellectually on an entire field.
FranzFerdiNaN 7y ago
Why not go one step further and only fund math, because in the end everything is math, right? No need to fund physics, which is just applied math. No need to fund medicine, which is just applied math. And medicine is a field filled with terrible studes that cant be replicated, so clearly it cant be good for anything.

abdullahkhalids 7y ago

> After a lot of arguing and furious paper writing in the late 80s-90s, Can you cite at least one paper from this series?

AlotOfReading 7y ago
My background is in archaeology, so that's what I'll discuss. The other social sciences had similar debates around the same time, plus or minus a few years. If you're looking to understand one perspective from this period, I'd actually recommend Binford's Debating Archaeology [1]. This book summarizes a whole slew of arguments Binford had in the 80s with a number of other archaeologists, including his famous spate with Gould over the limits of empiricism. As it's a summary, it was obviously written after all of this was settled. Selected works from the era might be Hodder's Symbolic and Structural Archaeology[2] or kohl's "Limits to Post-Processual Archaeology" [3]. The former was essentially an opening salvo for this debate and the latter is an example of a the reactions to it. [1] Binford, L. R. (2016). Debating Archaeology: Updated Edition. Routledge. [2] Hodder, I. (1982). Symbolic and Structural Archaeology. [3] Kohl, P. L. (1993). Limits to a post-processual archaeology (or, The dangers of a new scholasticism). Archaeological Theory: who sets the agenda, 13-19.