4 ms·

Then it's outright fraud to present your results as reliable when there's radically uncontrolled variables that dominate the results.

sideshowb 9y ago
It would only be fraud if done with deliberate intent to deceive.
Cthulhu_ 9y ago
Just make sure to note the variables if you can list them. The main problem is media outlets taking the results as fact.
nonbel 9y ago
This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out. That said, you do learn not to act 100% confident. You need to say "This suggests that" instead of "This proves that", etc. Anyway, the solution to reliable data is simple: Have multiple competing groups try to repeat the work as closely as possible. Now that you have something to hang your hat on, you come up with some theory/model to explain the phenomenon. Even better, come up with multiple competing such models. From these you deduce some precise prediction that is checked against future data. All of this was well known a few generations ago...
- SomeStupidPoint 9y ago
  > This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out. Charlatan fortunetellers aren't owed careers as scientists -- if they can't produce reliable, predictive models because they utterly fail to control experiments,then they should have their careers destroyed. We've seen enough damage from corporate leaders and politicians relying on guides to reading chicken entrails masquerading as science. It's time to stop the fraud.
  - majewsky 9y ago
    Consider the consequences of what you're demanding: You'd basically outlaw any research on complex systems. Everything in biology, sociology, archeology, climate science, etc. Is that a reasonable response to the problem at hand?
    - nonbel 9y ago
      Why would it be outlawed?
    - SomeStupidPoint 9y ago
      No, I would bifurcate research into two classes -- Let's call it 'science' and 'hunches'. (I'm sure marketing people would think of a better name.) There's nothing stopping you from publishing hunches; there's nothing stopping the same journals from publishing science and hunches. It would just be the case that we'd treat publishing a hunch as science as fraud, because you're outright lying about how sure you are. It is the case that several fields would get (almost entirely) moved from science to hunches, but that would be good! The label will match what's inside. If we have this split in the product lines, if we have these two tiers (models where we're really sure and complex systems where we only have a working hunch), then we should just own it and put that on the tin. All I'm asking is people stop labeling composite wood as lumber -- and understandably, a lot of people selling composites act like that would make the sky fall. Hardly, there's a real demand for both products and both serve a real need. Let's just be honest in our marketing and stop conflating the two.
  - nonbel 9y ago
    Sure, just be aware of what you are saying. Over 99% of what gets published is like this. Generations have wasted their careers producing misinformation. It all needs to be redone since these practices began (in some areas of research this is at t=0). Etc.

sideshowb 9y ago

It would only be fraud if done with deliberate intent to deceive.

Cthulhu_ 9y ago

Just make sure to note the variables if you can list them. The main problem is media outlets taking the results as fact.

nonbel 9y ago

This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out. That said, you do learn not to act 100% confident. You need to say "This suggests that" instead of "This proves that", etc. Anyway, the solution to reliable data is simple: Have multiple competing groups try to repeat the work as closely as possible. Now that you have something to hang your hat on, you come up with some theory/model to explain the phenomenon. Even better, come up with multiple competing such models. From these you deduce some precise prediction that is checked against future data. All of this was well known a few generations ago...

SomeStupidPoint 9y ago
> This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out. Charlatan fortunetellers aren't owed careers as scientists -- if they can't produce reliable, predictive models because they utterly fail to control experiments,then they should have their careers destroyed. We've seen enough damage from corporate leaders and politicians relying on guides to reading chicken entrails masquerading as science. It's time to stop the fraud.
- majewsky 9y ago
  Consider the consequences of what you're demanding: You'd basically outlaw any research on complex systems. Everything in biology, sociology, archeology, climate science, etc. Is that a reasonable response to the problem at hand?
  - nonbel 9y ago
    Why would it be outlawed?
  - SomeStupidPoint 9y ago
    No, I would bifurcate research into two classes -- Let's call it 'science' and 'hunches'. (I'm sure marketing people would think of a better name.) There's nothing stopping you from publishing hunches; there's nothing stopping the same journals from publishing science and hunches. It would just be the case that we'd treat publishing a hunch as science as fraud, because you're outright lying about how sure you are. It is the case that several fields would get (almost entirely) moved from science to hunches, but that would be good! The label will match what's inside. If we have this split in the product lines, if we have these two tiers (models where we're really sure and complex systems where we only have a working hunch), then we should just own it and put that on the tin. All I'm asking is people stop labeling composite wood as lumber -- and understandably, a lot of people selling composites act like that would make the sky fall. Hardly, there's a real demand for both products and both serve a real need. Let's just be honest in our marketing and stop conflating the two.
- nonbel 9y ago
  Sure, just be aware of what you are saying. Over 99% of what gets published is like this. Generations have wasted their careers producing misinformation. It all needs to be redone since these practices began (in some areas of research this is at t=0). Etc.