3 ms·

Is it just me, or is there a lot less "bad science" being done in chemistry and (testable) physics? Maybe one way to decide if a discipline is "scientific" in

Is it just me, or is there a lot less "bad science" being done in chemistry and (testable) physics? Maybe one way to decide if a discipline is "scientific" in nature is how well the scientific process actually works within that discipline? And if we discover evidence that the scientific process doesn't work well in a particular discipline (for whatever reason—doesn't matter why, actually), maybe we should stop calling that discipline a "science" and come up with some other descriptor?

mtriano 10y ago
What evidence/data have you used to form your opinion in regards to hard sciences? I don't necessarily disagree, but considering the subject matter, I feel compelled to press you for a data-supported justification.
- amatic 10y ago
  Same question. I would assume this is partially based on replicability, or doing the study over and getting the same results. Recently, there was an attempt to replicate major studies in psychology and only about 36% were replicated. I think something similar was in medicine.
- sevensor 10y ago
  I've seen copious bad papers in EE and MechE journals. I'm not qualified to evaluate chem or physics research, but you'd think if any field would be hard to fudge, it would be engineering. Turns out this is not the case. The bad engineering papers I've seen tend to draw overly-broad conclusions while failing to document the methods adequately.
forgetsusername 10y ago
Changing the descriptor would be a classic case of bike-shedding.
- erichocean 10y ago
  > Changing the descriptor would be a classic case of bike-shedding. Strongly disagree. Things that are "scientific" get way more acceptance by the public then things that are "hey, that's just, like, your opinion, man." For instance, in what sense at all is "political science" an actual science in the sense of chemistry or physics? It seems to me that it's an entirely different thing, and so I'm not surprised at all when a political science "result" fails to replicate or is outright bullshit. Science has come to mean, in many cases, "area you can get a PhD in" vs. what we tend to act like it means, which is "area where the scientific method applies." I guess I'd like to see the word science only used for the latter meaning if we're going to continue to give "scientific" disciplines extra respect. If instead we want to continue to use "science" to denote "anything you can get a PhD in", then we should seriously degrade or eliminate altogether our instinct to trust "scientific" results.
GlennS 10y ago
I disagree. People working in the 'soft sciences' should still be aiming to work scientifically, so we should continue to describe them as such. In practice it is very hard to measure up to the standards of good science in a lot of disciplines, where there's enough complexity to make constructing good experiments difficult. I'm thinking in these rough categories: 1. Real science that demonstrably works now. 2. An attempt at science that may become (1) in the future, but doesn't quite work yet. 3. Pseudo-science. The trappings of science without the intellectual honesty. 4. Claims about truth that don't pretend to be scientific at all. So, for category (2), we can have some respect for the scientific integrity of the practitioners, while at the same time being skeptical of the results and accepted theories for the time being. There's also the side-matter of avoiding the engineer's sneer towards people who try to understand things but refuse to learn the maths. I don't think it's a helpful attitude, although I don't know what the right fix is. I do partially agree with you though. It would be good to make sure that people making political decisions based on economics or sociology do understand that those subjects aren't quite ready yet.
- erichocean 10y ago
  > 1. Real science that demonstrably works now. One of the things I noticed in college is that the classes I took in this category never actually taught "the scientific method." It was only ever in the so-called "soft" sciences that we got the scientific method lecture about creating a hypothesis, doing some tests to verify, checking results, etc.—usually in the first week. So maybe a way to identify classes still in your category "2" is to see if they explicitly teach the scientific method or not. If they do? Category 2. If they're obviously scientific and don't teach it? Category 1. (I've also noticed that an easy way to identify a crackpot on the Internet is to see if they trot out the scientific method discussion in the first 1000 words. If they do? Crackpot.)
bbctol 10y ago
There are lots of reasons for higher failure rates in one discipline than another that have nothing to do with how "scientific" they are. Cancer biology is indisputably a science, and has a higher rate of non-replicable results than non-organic chemistry; that's because it's much more complicated, not because it's some sort of unscientific philosophizing. Psychology, which gets the most grief from hard physical STEM types whenever this issue comes up, is hard to reproduce because it's hard in general. It's the study of the most complex object we know about, and we can only access it through indirect and creative means. Small differences in study design can have large impacts on outcomes because the brain is sensitive, and complicated, and often counter-intuitive. It's not that there's more "bad science" being done in some disciplines, it's that those disciplines require much more scrutiny towards their methods and inferences, because they're harder to handle. But they're still science, and it would be a horrible disservice to abandon the scientific method for them just because it's hard.

mtriano 10y ago

What evidence/data have you used to form your opinion in regards to hard sciences? I don't necessarily disagree, but considering the subject matter, I feel compelled to press you for a data-supported justification.

amatic 10y ago
Same question. I would assume this is partially based on replicability, or doing the study over and getting the same results. Recently, there was an attempt to replicate major studies in psychology and only about 36% were replicated. I think something similar was in medicine.
sevensor 10y ago
I've seen copious bad papers in EE and MechE journals. I'm not qualified to evaluate chem or physics research, but you'd think if any field would be hard to fudge, it would be engineering. Turns out this is not the case. The bad engineering papers I've seen tend to draw overly-broad conclusions while failing to document the methods adequately.

forgetsusername 10y ago

Changing the descriptor would be a classic case of bike-shedding.

erichocean 10y ago
> Changing the descriptor would be a classic case of bike-shedding. Strongly disagree. Things that are "scientific" get way more acceptance by the public then things that are "hey, that's just, like, your opinion, man." For instance, in what sense at all is "political science" an actual science in the sense of chemistry or physics? It seems to me that it's an entirely different thing, and so I'm not surprised at all when a political science "result" fails to replicate or is outright bullshit. Science has come to mean, in many cases, "area you can get a PhD in" vs. what we tend to act like it means, which is "area where the scientific method applies." I guess I'd like to see the word science only used for the latter meaning if we're going to continue to give "scientific" disciplines extra respect. If instead we want to continue to use "science" to denote "anything you can get a PhD in", then we should seriously degrade or eliminate altogether our instinct to trust "scientific" results.

GlennS 10y ago

I disagree. People working in the 'soft sciences' should still be aiming to work scientifically, so we should continue to describe them as such. In practice it is very hard to measure up to the standards of good science in a lot of disciplines, where there's enough complexity to make constructing good experiments difficult. I'm thinking in these rough categories: 1. Real science that demonstrably works now. 2. An attempt at science that may become (1) in the future, but doesn't quite work yet. 3. Pseudo-science. The trappings of science without the intellectual honesty. 4. Claims about truth that don't pretend to be scientific at all. So, for category (2), we can have some respect for the scientific integrity of the practitioners, while at the same time being skeptical of the results and accepted theories for the time being. There's also the side-matter of avoiding the engineer's sneer towards people who try to understand things but refuse to learn the maths. I don't think it's a helpful attitude, although I don't know what the right fix is. I do partially agree with you though. It would be good to make sure that people making political decisions based on economics or sociology do understand that those subjects aren't quite ready yet.

erichocean 10y ago
> 1. Real science that demonstrably works now. One of the things I noticed in college is that the classes I took in this category never actually taught "the scientific method." It was only ever in the so-called "soft" sciences that we got the scientific method lecture about creating a hypothesis, doing some tests to verify, checking results, etc.—usually in the first week. So maybe a way to identify classes still in your category "2" is to see if they explicitly teach the scientific method or not. If they do? Category 2. If they're obviously scientific and don't teach it? Category 1. (I've also noticed that an easy way to identify a crackpot on the Internet is to see if they trot out the scientific method discussion in the first 1000 words. If they do? Crackpot.)

bbctol 10y ago

There are lots of reasons for higher failure rates in one discipline than another that have nothing to do with how "scientific" they are. Cancer biology is indisputably a science, and has a higher rate of non-replicable results than non-organic chemistry; that's because it's much more complicated, not because it's some sort of unscientific philosophizing. Psychology, which gets the most grief from hard physical STEM types whenever this issue comes up, is hard to reproduce because it's hard in general. It's the study of the most complex object we know about, and we can only access it through indirect and creative means. Small differences in study design can have large impacts on outcomes because the brain is sensitive, and complicated, and often counter-intuitive. It's not that there's more "bad science" being done in some disciplines, it's that those disciplines require much more scrutiny towards their methods and inferences, because they're harder to handle. But they're still science, and it would be a horrible disservice to abandon the scientific method for them just because it's hard.