4 ms·

I agree with this article, but think that by singling out social sciences it fails to reveal the bigger picture: that all scientific fields are subject to these

by wmacmil 12y ago

I agree with this article, but think that by singling out social sciences it fails to reveal the bigger picture: that all scientific fields are subject to these same mistakes. I just started working in a research lab at a hospital that creates finite element cardiac models based on data taken during heart surgeries on sheep along with MRI images taken at various intervals pre and post surgery. Although I'm still new to this position, it seems that our own methods are subject to just as much deception. We basically want our models to coordinate with the actual heartbeat at only two exact moments, during the beginning of contraction and relaxation. If I've understood what's been done before, modeling these two brief periods during a single heart beat are all that are needed for publication. I bring this up not to criticize my lab, obviously our work is meant to be a progression towards getting more and more accurate models. I just think that it shows that even something that is considered hard science is subject to many of the same faults as anything else. There are so many parameters and considerations to take into account that I don't think the end goal is to build a comprehensive theory that explains computational modeling of physiological function in a similar way to how Newton's laws predict the motions of the planets. The goal is simply to create a model that works for the purposes of helping diagnose and treat people more accurately. It seems that a crisis is imminent in the coming age of computational, statistical, and mathematical applications to all fields where researchers are not properly taught to distinguish between data science and building actual theories. Just as there is a humungous gap between using a computer and actually coding, there is an equivalent difference between being able to collect & analyse data with a computer, and able to actually build a substantial theory that can describe a vast number of phenomena and result.

privong 12y ago
> I agree with this article, but think that by singling out social sciences it fails to reveal the bigger picture: that all scientific fields are subject to these same mistakes. I actually prefer them writing about a single (or a group of related) field(s) of study. There have been lots of articles asserting that all of science has these issues, but it's not clear to me that they all suffer to the same degree. For example, It's not clear to me that physics or astronomy suffer as from this type of behavior. I'm sure it does happen, but it doesn't seem to be as pervasive as in the social sciences and medicine. For social sciences it seems like it is happening often enough that it's affecting the trajectory of the field. To my knowlege, this isn't the case for most of physics and astronomy (for example). Also, a lot (but not all) of the "mistakes" noted in the article were deliberate fabrications or unethical tampering of data. So I think labeling them as "mistakes" risks being too forgiving.
- tedks 12y ago
  Psychology tends to be picked on more because it's easier to understand and communicate. Researchers in psychology tend to be in areas that have names that are meaningful to humans (like "short term memory" or "interpersonal violence") rather than jargon ("systems" or "finite mass spectroscopy"). Really, psychology has the bad luck of having jargon that maps to lay knowledge -- imperfectly, but the reporters don't care. Overall, most scientists are ethical. Very few people are deliberately falsifying their data. But in every field, the incentives are not set up to do good science. That's a fact, and until it's understood there won't be any progress. Until then, you'll just see a slow grind of academia eating itself.
  - privong 12y ago
    > But in every field, the incentives are not set up to do good science. That's a fact, and until it's understood there won't be any progress. Until then, you'll just see a slow grind of academia eating itself. I think this depends on what you mean by "doing good science". I agree that incentives in every field are not set up to do the _best_ science, but in many fields the incentives are not such that people are drawn to doing _bad_ science. So if academia is "eating itself", I don't think it can be universally attributed to incentives regarding good vs bad science.
    - tedks 12y ago
      No, it's the same in every field, because the process of getting tenure in about every field is the same: publish lots of times. "Lots" might be different in different fields, but it still comes down to "publish a lot." Publishing a lot is actually a lot closer to incentivizing bad science than not. I'd encourage you to look past your pre-existing notions of what makes a "hard" science branch and actually talk to people who are the boots on the ground about what is really going on in academia.

by wmacmil 12y ago

privong 12y ago

> I agree with this article, but think that by singling out social sciences it fails to reveal the bigger picture: that all scientific fields are subject to these same mistakes. I actually prefer them writing about a single (or a group of related) field(s) of study. There have been lots of articles asserting that all of science has these issues, but it's not clear to me that they all suffer to the same degree. For example, It's not clear to me that physics or astronomy suffer as from this type of behavior. I'm sure it does happen, but it doesn't seem to be as pervasive as in the social sciences and medicine. For social sciences it seems like it is happening often enough that it's affecting the trajectory of the field. To my knowlege, this isn't the case for most of physics and astronomy (for example). Also, a lot (but not all) of the "mistakes" noted in the article were deliberate fabrications or unethical tampering of data. So I think labeling them as "mistakes" risks being too forgiving.

tedks 12y ago
Psychology tends to be picked on more because it's easier to understand and communicate. Researchers in psychology tend to be in areas that have names that are meaningful to humans (like "short term memory" or "interpersonal violence") rather than jargon ("systems" or "finite mass spectroscopy"). Really, psychology has the bad luck of having jargon that maps to lay knowledge -- imperfectly, but the reporters don't care. Overall, most scientists are ethical. Very few people are deliberately falsifying their data. But in every field, the incentives are not set up to do good science. That's a fact, and until it's understood there won't be any progress. Until then, you'll just see a slow grind of academia eating itself.
- privong 12y ago
  > But in every field, the incentives are not set up to do good science. That's a fact, and until it's understood there won't be any progress. Until then, you'll just see a slow grind of academia eating itself. I think this depends on what you mean by "doing good science". I agree that incentives in every field are not set up to do the _best_ science, but in many fields the incentives are not such that people are drawn to doing _bad_ science. So if academia is "eating itself", I don't think it can be universally attributed to incentives regarding good vs bad science.
  - tedks 12y ago
    No, it's the same in every field, because the process of getting tenure in about every field is the same: publish lots of times. "Lots" might be different in different fields, but it still comes down to "publish a lot." Publishing a lot is actually a lot closer to incentivizing bad science than not. I'd encourage you to look past your pre-existing notions of what makes a "hard" science branch and actually talk to people who are the boots on the ground about what is really going on in academia.