4 ms·

Science is not when you publish something. Science is when at least two independent labs can reproduce your published result.

aldoushuxley001 8y ago
Why two and not three or one?
- skh 8y ago
  If two independent labs can reproduce the result it is extremely unlikely to be the result of false positives. The difference between having two versus three independent verifications is minuscule. A standard has to be set and 2 is a good enough standard.
- daveguy 8y ago
  The more the better. You'd really need a statistically significant number of labs repeating it to be sure. If the underlying study is scientifically sound then a single repeat is good, two repeats is better. 20 repeats is even better. Good luck getting 1 or 2, unless the study is an important work that introduces a useful new technique or is the basis for additional studies. Also good luck knowing whether 2 labs failed for every 1 published. Science is hard. Reporting needs to be improved and there should be grants available to fund repeat studies.
- v_lisivka 8y ago
  It's easy to keep secret when you only one who knows truth. It's also easy for two, because you immediately know who to blame if secret is revealed, so you can keep pressure. But situation changes for three, because you cannot know who revealed secret, so it's much easier to crack. Imagine, I will report today that I'm able to determine temperature of the physical vacuum (e.g. using double slit experiment), and then able to drop it, i.e. cool vacuum below absolute zero. Will you trust me? Of course, not. Then when you will trust me?
mehrdadn 8y ago
The issue I see with this is that it implies anyone who isn't doing work high-profile enough to get independent labs to try to reproduce it isn't doing science.
- v_lisivka 8y ago
  IHMO, it's correct. They are just sitting on shoulders of giants.
  - mehrdadn 8y ago
    And you're not a developer/engineer until 2 companies independently use your work, right?
yurlungur 8y ago
While I have a similar disillusion with the academia, I think sometimes people misinterpret the difficulty with reproducibility, especially on a field they are not familiar with. It seems to me it's imprudent and a bit disrespectful to jump to the gun when a certain research fail to get reproduced, especially when it comes to biology, because the experimental procedure can be incredibly complex and prone to all kinds of errors. I know of cases where even within a group only one person can successfully perform a certain protocol with a good enough success rate and it takes a lot of learning to get on their level. Without expertise in the field and knowledge of how the reproduction is done, it's really hard to examine all the data and judge the integrity of the science behind it. To me the much bigger problem is how so much of the research is warped by what gets funding, fast.
- v_lisivka 8y ago
  Wrong result or no result is result too. With help of those who tried to reproduce and failed, it's easier to find missing gaps in knowledge and fill them. Otherwise this knowledge will be lost.
analog31 8y ago
So, build two more LHC's. ;-) In my view it should be a judgment call whether a particular result needs to be replicated. I'm aware of the so called replication crisis. But adding more arbitrary rules will simply invite people to figure out how to game those rules, if they were gaming the original rules (deliberately or unwittingly).
- v_lisivka 8y ago
  LHC is not the first collider in the world, so results from previous experiments are used to calibrate LHC.
  - analog31 8y ago
    Yes, this seems like a much less wasteful, and possibly more fruitful approach. Rather than running identical experiments, one can perform experiments that test different but overlapping aspects of the underlying theory. I suspect this kind of "replication" is done all the time in physics, which has led to a robust knowledge base in spite of less than ideal rigor within any individual experiment.

aldoushuxley001 8y ago

Why two and not three or one?

skh 8y ago
If two independent labs can reproduce the result it is extremely unlikely to be the result of false positives. The difference between having two versus three independent verifications is minuscule. A standard has to be set and 2 is a good enough standard.
daveguy 8y ago
The more the better. You'd really need a statistically significant number of labs repeating it to be sure. If the underlying study is scientifically sound then a single repeat is good, two repeats is better. 20 repeats is even better. Good luck getting 1 or 2, unless the study is an important work that introduces a useful new technique or is the basis for additional studies. Also good luck knowing whether 2 labs failed for every 1 published. Science is hard. Reporting needs to be improved and there should be grants available to fund repeat studies.
v_lisivka 8y ago
It's easy to keep secret when you only one who knows truth. It's also easy for two, because you immediately know who to blame if secret is revealed, so you can keep pressure. But situation changes for three, because you cannot know who revealed secret, so it's much easier to crack. Imagine, I will report today that I'm able to determine temperature of the physical vacuum (e.g. using double slit experiment), and then able to drop it, i.e. cool vacuum below absolute zero. Will you trust me? Of course, not. Then when you will trust me?

mehrdadn 8y ago

The issue I see with this is that it implies anyone who isn't doing work high-profile enough to get independent labs to try to reproduce it isn't doing science.

v_lisivka 8y ago
IHMO, it's correct. They are just sitting on shoulders of giants.
- mehrdadn 8y ago
  And you're not a developer/engineer until 2 companies independently use your work, right?

yurlungur 8y ago

While I have a similar disillusion with the academia, I think sometimes people misinterpret the difficulty with reproducibility, especially on a field they are not familiar with. It seems to me it's imprudent and a bit disrespectful to jump to the gun when a certain research fail to get reproduced, especially when it comes to biology, because the experimental procedure can be incredibly complex and prone to all kinds of errors. I know of cases where even within a group only one person can successfully perform a certain protocol with a good enough success rate and it takes a lot of learning to get on their level. Without expertise in the field and knowledge of how the reproduction is done, it's really hard to examine all the data and judge the integrity of the science behind it. To me the much bigger problem is how so much of the research is warped by what gets funding, fast.

v_lisivka 8y ago
Wrong result or no result is result too. With help of those who tried to reproduce and failed, it's easier to find missing gaps in knowledge and fill them. Otherwise this knowledge will be lost.

analog31 8y ago

So, build two more LHC's. ;-) In my view it should be a judgment call whether a particular result needs to be replicated. I'm aware of the so called replication crisis. But adding more arbitrary rules will simply invite people to figure out how to game those rules, if they were gaming the original rules (deliberately or unwittingly).

v_lisivka 8y ago
LHC is not the first collider in the world, so results from previous experiments are used to calibrate LHC.
- analog31 8y ago
  Yes, this seems like a much less wasteful, and possibly more fruitful approach. Rather than running identical experiments, one can perform experiments that test different but overlapping aspects of the underlying theory. I suspect this kind of "replication" is done all the time in physics, which has led to a robust knowledge base in spite of less than ideal rigor within any individual experiment.