3 ms·

If your data is not good enough, get better data. Don't abandon objectivity. I think this is why a lot of people don't take the social sciences seriously, there

If your data is not good enough, get better data. Don't abandon objectivity. I think this is why a lot of people don't take the social sciences seriously, there's an extreme resistance to just doing objective, data driven research (and the lack of reproducibility that comes with that).

jftj 7y ago
It's not that "the data is not good enough". It's that it's easy to obfuscate results by hiding bias in the data, even if unintentionally. Even in the "hard" sciences. Or have we forgotten about some of the awful research being done in machine learning? Remember the one about identifying future criminals from photos?
ajb 7y ago
It took 200 years for the hard sciences to figure out the right way to 'do' objectivity, and arguably it's still evolving (see 'Objectivity'[1] by Daston and Galison, nice summary here: https://scienceobjectivity.weebly.com/daston--galison.html https://scienceobjectivity.weebly.com/daston--galison.html). The social sciences have additional problems, saying 'just copy the hard sciences' isn't reasonable. In fact, the idea that objectivity can be done in a simple mechanical way doesn't work for the hard sciences either, and was rejected by them in the early 20th, because it turns out that interpreting the evidence objectively still requires an educated eye. If you've ever had the good fortune (or misfortune, considering the circs) to see one of your own x-rays, you'll know how much you have to take the word of the radiographer for what the photo actually means. Sure, eventually, after much debate, a mechanical formula for interpreting the photo may be developed. Radiographers are probably due to be replaced by AI any day now - but their judgements will have formed the labels of the training data. At the cutting edge of the sciences, you don't have any of that yet. [1] that book was expanded from their earlier paper which can be found here: http://cspeech.ucd.ie/Fred/docs/Galison.pdf http://cspeech.ucd.ie/Fred/docs/Galison.pdf
AlotOfReading 7y ago
Again, the issue isn't one of methodology or technology. It's inherent to the research process. Let's do a thought experiment and imagine two identical research projects done in parallel universes. Everything is the same between these two universes except that the researchers are different people. In some fields, you'd probably expect the results to be the same. The researcher shouldn't affect objective results after all. In a social science setting, the results may not be the same even if both researchers used exactly the same research design on exactly the same subjects at exactly the same point in their lives. It should be very clear by this point in the thought experiment that "objectivity" is not really possible under those conditions. You need some theory to deal with this issue, but honestly it's manageable for the most part. Now, the real kicker is that researchers affect their research way more than this. They actually design the project and constantly make decisions / interpretations throughout. These decisions and interpretations are subjective by definition. Even if we had a magical way to get objective data, are you really still willing to call it objective after a dirty, subjective human has so much as looked at it, let alone done the sort of filtering and interpretation inherent to academic research in all fields?
- malvosenior 7y ago
  I think it says a lot about the value of the social “sciences” that they can’t be backed by data and aren’t reproducible. It seems that maybe they can be data driven but the results aren’t what the traditional practitioners of the field like to see. I think we’ll see the role they historically played replaced by machine learning which has more predictive power simply because it’s backed by reality.
- naasking 7y ago
  > The researcher shouldn't affect objective results after all. In a social science setting, the results may not be the same even if both researchers used exactly the same research design on exactly the same subjects at exactly the same point in their lives. This is true of physics as well. Measurements in quantum experiments are contextual, in that the experimental setup interferes with system being measured. Measurements conform to statistical distributions. The difference is that physicists took the time to quantify how measurements influence the system, thus generating a practical theory. It's not clear to me that something similar couldn't be done here, even if human systems are in a sense, messier. In fact, it's already been done to some extent because we clearly know how the phrasing of some questions influence the answers. We also know from tragedies of justice [1] how important this subject is. Replication and a culture that encourages publishing negative results should be prerequisites for any field to qualify as a science. [1] https://en.m.wikipedia.org/wiki/Day-care_sex-abuse_hysteria https://en.m.wikipedia.org/wiki/Day-care_sex-abuse_hysteria
- choeger 7y ago
  > It should be very clear by this point in the thought experiment that "objectivity" is not really possible under those conditions. You need some theory to deal with this issue, but honestly it's manageable for the most part. Tbh, That sounds like a lame excuse. The logical conclusion is that science done that way will never produce truth[1]. In consequence we should stop listening to such "scientists". [1] As in "can be reproduced and likely gives the same result"

jftj 7y ago

It's not that "the data is not good enough". It's that it's easy to obfuscate results by hiding bias in the data, even if unintentionally. Even in the "hard" sciences. Or have we forgotten about some of the awful research being done in machine learning? Remember the one about identifying future criminals from photos?

ajb 7y ago

It took 200 years for the hard sciences to figure out the right way to 'do' objectivity, and arguably it's still evolving (see 'Objectivity'[1] by Daston and Galison, nice summary here: https://scienceobjectivity.weebly.com/daston--galison.html https://scienceobjectivity.weebly.com/daston--galison.html). The social sciences have additional problems, saying 'just copy the hard sciences' isn't reasonable. In fact, the idea that objectivity can be done in a simple mechanical way doesn't work for the hard sciences either, and was rejected by them in the early 20th, because it turns out that interpreting the evidence objectively still requires an educated eye. If you've ever had the good fortune (or misfortune, considering the circs) to see one of your own x-rays, you'll know how much you have to take the word of the radiographer for what the photo actually means. Sure, eventually, after much debate, a mechanical formula for interpreting the photo may be developed. Radiographers are probably due to be replaced by AI any day now - but their judgements will have formed the labels of the training data. At the cutting edge of the sciences, you don't have any of that yet. [1] that book was expanded from their earlier paper which can be found here: http://cspeech.ucd.ie/Fred/docs/Galison.pdf http://cspeech.ucd.ie/Fred/docs/Galison.pdf

AlotOfReading 7y ago

Again, the issue isn't one of methodology or technology. It's inherent to the research process. Let's do a thought experiment and imagine two identical research projects done in parallel universes. Everything is the same between these two universes except that the researchers are different people. In some fields, you'd probably expect the results to be the same. The researcher shouldn't affect objective results after all. In a social science setting, the results may not be the same even if both researchers used exactly the same research design on exactly the same subjects at exactly the same point in their lives. It should be very clear by this point in the thought experiment that "objectivity" is not really possible under those conditions. You need some theory to deal with this issue, but honestly it's manageable for the most part. Now, the real kicker is that researchers affect their research way more than this. They actually design the project and constantly make decisions / interpretations throughout. These decisions and interpretations are subjective by definition. Even if we had a magical way to get objective data, are you really still willing to call it objective after a dirty, subjective human has so much as looked at it, let alone done the sort of filtering and interpretation inherent to academic research in all fields?

malvosenior 7y ago
I think it says a lot about the value of the social “sciences” that they can’t be backed by data and aren’t reproducible. It seems that maybe they can be data driven but the results aren’t what the traditional practitioners of the field like to see. I think we’ll see the role they historically played replaced by machine learning which has more predictive power simply because it’s backed by reality.
naasking 7y ago
> The researcher shouldn't affect objective results after all. In a social science setting, the results may not be the same even if both researchers used exactly the same research design on exactly the same subjects at exactly the same point in their lives. This is true of physics as well. Measurements in quantum experiments are contextual, in that the experimental setup interferes with system being measured. Measurements conform to statistical distributions. The difference is that physicists took the time to quantify how measurements influence the system, thus generating a practical theory. It's not clear to me that something similar couldn't be done here, even if human systems are in a sense, messier. In fact, it's already been done to some extent because we clearly know how the phrasing of some questions influence the answers. We also know from tragedies of justice [1] how important this subject is. Replication and a culture that encourages publishing negative results should be prerequisites for any field to qualify as a science. [1] https://en.m.wikipedia.org/wiki/Day-care_sex-abuse_hysteria https://en.m.wikipedia.org/wiki/Day-care_sex-abuse_hysteria
choeger 7y ago
> It should be very clear by this point in the thought experiment that "objectivity" is not really possible under those conditions. You need some theory to deal with this issue, but honestly it's manageable for the most part. Tbh, That sounds like a lame excuse. The logical conclusion is that science done that way will never produce truth[1]. In consequence we should stop listening to such "scientists". [1] As in "can be reproduced and likely gives the same result"