3 ms·

If scientists are going to complain that's its too hard or too expensive to replicate their studies, then that just shows their work is BS.

by abnry 3y ago

alsodumb 3y ago
Nah, it doesn't. It just shows that it's time consuming and expensive to replicate their studies.
- abnry 3y ago
  If that's the case, then don't claim confidence in the work or make policy decisions based off of it. If there is no epistemological humility, then yes, it is still BS.
  - Levitz 3y ago
    If any study costs X, the study and the replication costs somewhere in the ballpark of 2*X. This is not trivial.
    - abnry 3y ago
      But this is science we are talking about. A one-off lucky novel result should not be good enough. Why should our standards and our funding be so low?
fodkodrasz 3y ago
I guess if software developers will complain that it's too hard or too expensive to thoroughly test their code to ensure exactly zero bugs at release[1], then that just shows their work is BS. [1]: if you have delivered telco code to Softbank you may have heard this sentence
- abnry 3y ago
  Replication is not the same thing as zero bugs in software.
  - fodkodrasz 3y ago
    https://www.merriam-webster.com/dictionary/analogy https://www.merriam-webster.com/dictionary/analogy
azan_ 3y ago
What if it REALLY is too expensive? You do realize that there are studies which literally cost millions of dollars? Getting funding for original studies is hard enough, good luck securing additional funds for replication.
snitty 3y ago
>If scientists are going to complain that's its too hard or too expensive to replicate their studies, then that just shows their work is BS. 1 mg of anti-rabbit antibody (a common thing to use in a lot of biology experiments) is $225 [1]. Outside of things like standard buffers and growth medium for prokaryotes, this is going to be the cheapest thing you use in an experiment. 1/10th of that amount for anti-flagellin antibody is $372. [2] A kit to prep a cell for RNA sequencing is $6-10 per use. That's JUST isolation of the RNA. Not including reverse transcribing it to cDNA for sequencing, or the sequencing itself. [3] Let's not even reach things like materials science where you may be working on an epitaxial growth paper, and there are only a handful of labs where they could even feasibly repeat the experiment. Or say something with a BSL-3 lab where there are literally only 15 labs in the US that could feasibly do the work, assuming they aren't working on their own stuff. [4] [1] - https://www.thermofisher.com/antibody/product/Goat-anti-Rabbit-IgG-H-L-Cross-Adsorbed-Secondary-Antibody-Polyclonal/A-11008 https://www.thermofisher.com/antibody/product/Goat-anti-Rabb... [2] https://www.invivogen.com/anti-flagellin https://www.invivogen.com/anti-flagellin [3] https://www.thermofisher.com/order/catalog/product/12183018A https://www.thermofisher.com/order/catalog/product/12183018A [4] https://www.niaid.nih.gov/research/tufts-regional-biocontainment-lab https://www.niaid.nih.gov/research/tufts-regional-biocontain...
Maxion 3y ago
Something in switzerland called the Large Hadron Collider comes to mind. I guess we should not talk about the Higgs before someone else builds a second one and replicates the papers.
- abnry 3y ago
  Physics is generally better since they have good statistical models and can get six sigma (or whatever) results. And replication can be done by the same party (although an independent party is better), and that may mean many trials. And do we even set policy based on existence or non-existent of higgs bosons? I am particularly unhappy with soft sciences in terms of replication.

If scientists are going to complain that's its too hard or too expensive to replicate their studies, then that just shows their work is BS.

by abnry 3y ago

If scientists are going to complain that's its too hard or too expensive to replicate their studies, then that just shows their work is BS.

alsodumb 3y ago

Nah, it doesn't. It just shows that it's time consuming and expensive to replicate their studies.

abnry 3y ago
If that's the case, then don't claim confidence in the work or make policy decisions based off of it. If there is no epistemological humility, then yes, it is still BS.
- Levitz 3y ago
  If any study costs X, the study and the replication costs somewhere in the ballpark of 2*X. This is not trivial.
  - abnry 3y ago
    But this is science we are talking about. A one-off lucky novel result should not be good enough. Why should our standards and our funding be so low?

fodkodrasz 3y ago

I guess if software developers will complain that it's too hard or too expensive to thoroughly test their code to ensure exactly zero bugs at release[1], then that just shows their work is BS. [1]: if you have delivered telco code to Softbank you may have heard this sentence

abnry 3y ago
Replication is not the same thing as zero bugs in software.
- fodkodrasz 3y ago
  https://www.merriam-webster.com/dictionary/analogy https://www.merriam-webster.com/dictionary/analogy

azan_ 3y ago

What if it REALLY is too expensive? You do realize that there are studies which literally cost millions of dollars? Getting funding for original studies is hard enough, good luck securing additional funds for replication.

snitty 3y ago

>If scientists are going to complain that's its too hard or too expensive to replicate their studies, then that just shows their work is BS. 1 mg of anti-rabbit antibody (a common thing to use in a lot of biology experiments) is $225 [1]. Outside of things like standard buffers and growth medium for prokaryotes, this is going to be the cheapest thing you use in an experiment. 1/10th of that amount for anti-flagellin antibody is $372. [2] A kit to prep a cell for RNA sequencing is $6-10 per use. That's JUST isolation of the RNA. Not including reverse transcribing it to cDNA for sequencing, or the sequencing itself. [3] Let's not even reach things like materials science where you may be working on an epitaxial growth paper, and there are only a handful of labs where they could even feasibly repeat the experiment. Or say something with a BSL-3 lab where there are literally only 15 labs in the US that could feasibly do the work, assuming they aren't working on their own stuff. [4] [1] - https://www.thermofisher.com/antibody/product/Goat-anti-Rabbit-IgG-H-L-Cross-Adsorbed-Secondary-Antibody-Polyclonal/A-11008 https://www.thermofisher.com/antibody/product/Goat-anti-Rabb... [2] https://www.invivogen.com/anti-flagellin https://www.invivogen.com/anti-flagellin [3] https://www.thermofisher.com/order/catalog/product/12183018A https://www.thermofisher.com/order/catalog/product/12183018A [4] https://www.niaid.nih.gov/research/tufts-regional-biocontainment-lab https://www.niaid.nih.gov/research/tufts-regional-biocontain...

Maxion 3y ago

Something in switzerland called the Large Hadron Collider comes to mind. I guess we should not talk about the Higgs before someone else builds a second one and replicates the papers.

abnry 3y ago
Physics is generally better since they have good statistical models and can get six sigma (or whatever) results. And replication can be done by the same party (although an independent party is better), and that may mean many trials. And do we even set policy based on existence or non-existent of higgs bosons? I am particularly unhappy with soft sciences in terms of replication.