5 ms·

Here's a repeatable experiment in the social sciences: Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA's Control Randomized Experime

Here's a repeatable experiment in the social sciences: Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA's Control Randomized Experiment Author(s): Paul Gertler Source: The American Economic Review, Vol. 94, No. 2, Papers and Proceedings of the One Hundred Sixteenth Annual Meeting of the American Economic Association San Diego, CA, January 3-5, 2004 (May, 2004), pp. 336-341 From Section IV (you're gonna have to Sci-Hub it): "The randomization and the fact that the control and treatment samples are well balanced in the observed characteristics imply that a simple comparison of mean outcomes post-intervention will likely provide an unbiased estimate of program impacts. However, I also control for other observed socioeconomic characteristics in order to reduce idiosyncratic variation and to improve the power of the estimates." Is this really more sus than a cell bio paper? Are the conclusions so weak that it's not real science? Were you just being sarcastic and it went over my head?

chordalkeyboard 6y ago
It’s not strictly repeatable because those experimental subjects are not identical to the ones you would use to repeat the experiment. Not only are they morphologically different specimens, but they are culturally/behaviorally different as well. I’m not being sarcastic, I’m being strict about what constitutes a basis for a valid generalization. > Is this really more sus than a cell bio paper? Are the conclusions so weak that it's not real science? Its not necessarily that its more suspect or weaker, but that it is in a different epistemological category. Its a different kind of evidence. When you inject particles into a cloud chamber, you believe with very high confidence that those particles have fundamentally identical properties as the other particles of the same kind. We know that humans are not alike each other in the same way as particles. Therefore we are not permitted to treat them that way.
- phobosanomaly 6y ago
  What's the purpose of being so strict about what constitutes that basis for a valid generalization? What does it ultimately accomplish?
  - chordalkeyboard 6y ago
    Epistemological hygeine. It encourages people to be more careful about ensuring that they understand the degree to which a particular evidence complex supports a belief, and awareness of the corresponding possibility that the evidence is an artifact of a biased sample.
    - phobosanomaly 6y ago
      Fair enough. So, in the category of science, that leaves Physics, and what else? What would you call all of what we now call the biological, social, medical, sciences at the moment? What would be your proposal for a new category for everything that uses the scientific method, but that may have variations in the individual elements of the treatment and control groups? Maybe it would be more practical to change the name of Physics to something else, rather than re-categorizing all of the other fields. We could refer to physics as 'Invariant Studies' or something like that, and keep calling everything else science. That way might be easier across the board. I would propose as an alternative that we work to improve general scientific literacy so that the general public might be more familiar with the methods used in each field. For example, if more people knew how regression analysis worked they might be better able to understand it as the foundation for many of the conclusions in the social sciences, and so might be better able to understand the generalizability and potential limitations of what they are seeing when they are presented with a conclusion in one of these fields.
      - chordalkeyboard 6y ago
        Physics and chemistry. Its possible to do science in biology with things like knockout mice and physical limits. Other things are more a matter of classification and interpretation and it should be clear that they are socially constructed to a greater degree than physics or chemistry. > What would you call all of what we now call the biological, social, medical, sciences at the moment? I don’t know, I was originally defending the comment on the basis of “real science” so I don’t see a problem calling medicine an applied science. > What would be your proposal for a new category for everything that uses the scientific method, but that may have variations in the individual elements of the treatment and control groups? See, what I mean is actually a point about the scientific method. Its not really possible to use the scientific method without ensuring that your experimental subjects are indistinguishable. The reason I’m gatekeeping is because its not the same method, it would be pseudoscience. Like doing experiments on football games. > I would propose as an alternative that we work to improve general scientific literacy so that the general public might be more familiar with the methods used in each field. For example, if more people knew how regression analysis worked they might be better able to understand it as the foundation for many of the conclusions in the social sciences, and so might be better able to understand the generalizability and potential limitations of what they are seeing when they are presented with a conclusion in one of these fields. I think it would be great if people understood these things better, I guess I see this distinction as a way of articulating those fundamental differences in the mind of the public.

chordalkeyboard 6y ago

It’s not strictly repeatable because those experimental subjects are not identical to the ones you would use to repeat the experiment. Not only are they morphologically different specimens, but they are culturally/behaviorally different as well. I’m not being sarcastic, I’m being strict about what constitutes a basis for a valid generalization. > Is this really more sus than a cell bio paper? Are the conclusions so weak that it's not real science? Its not necessarily that its more suspect or weaker, but that it is in a different epistemological category. Its a different kind of evidence. When you inject particles into a cloud chamber, you believe with very high confidence that those particles have fundamentally identical properties as the other particles of the same kind. We know that humans are not alike each other in the same way as particles. Therefore we are not permitted to treat them that way.

phobosanomaly 6y ago
What's the purpose of being so strict about what constitutes that basis for a valid generalization? What does it ultimately accomplish?
- chordalkeyboard 6y ago
  Epistemological hygeine. It encourages people to be more careful about ensuring that they understand the degree to which a particular evidence complex supports a belief, and awareness of the corresponding possibility that the evidence is an artifact of a biased sample.
  - phobosanomaly 6y ago
    Fair enough. So, in the category of science, that leaves Physics, and what else? What would you call all of what we now call the biological, social, medical, sciences at the moment? What would be your proposal for a new category for everything that uses the scientific method, but that may have variations in the individual elements of the treatment and control groups? Maybe it would be more practical to change the name of Physics to something else, rather than re-categorizing all of the other fields. We could refer to physics as 'Invariant Studies' or something like that, and keep calling everything else science. That way might be easier across the board. I would propose as an alternative that we work to improve general scientific literacy so that the general public might be more familiar with the methods used in each field. For example, if more people knew how regression analysis worked they might be better able to understand it as the foundation for many of the conclusions in the social sciences, and so might be better able to understand the generalizability and potential limitations of what they are seeing when they are presented with a conclusion in one of these fields.
    - chordalkeyboard 6y ago
      Physics and chemistry. Its possible to do science in biology with things like knockout mice and physical limits. Other things are more a matter of classification and interpretation and it should be clear that they are socially constructed to a greater degree than physics or chemistry. > What would you call all of what we now call the biological, social, medical, sciences at the moment? I don’t know, I was originally defending the comment on the basis of “real science” so I don’t see a problem calling medicine an applied science. > What would be your proposal for a new category for everything that uses the scientific method, but that may have variations in the individual elements of the treatment and control groups? See, what I mean is actually a point about the scientific method. Its not really possible to use the scientific method without ensuring that your experimental subjects are indistinguishable. The reason I’m gatekeeping is because its not the same method, it would be pseudoscience. Like doing experiments on football games. > I would propose as an alternative that we work to improve general scientific literacy so that the general public might be more familiar with the methods used in each field. For example, if more people knew how regression analysis worked they might be better able to understand it as the foundation for many of the conclusions in the social sciences, and so might be better able to understand the generalizability and potential limitations of what they are seeing when they are presented with a conclusion in one of these fields. I think it would be great if people understood these things better, I guess I see this distinction as a way of articulating those fundamental differences in the mind of the public.