4 ms·

Very often, one reports that a particular phenomenon or result reported in previously could not be reproduced in a different experiment. This is just reporting

Very often, one reports that a particular phenomenon or result reported in previously could not be reproduced in a different experiment. This is just reporting that that particular result may be complex and context-specific. This is a negative citation but has no negative connotation, it does not even imply disagreement. Only in relatively rare cases where an exact experiment is repeated in the same context and the stated result could not be reproduced, it is an attack on the credibility of original work and is truly a 'negative' (as in bad) citation. The issue here is that the 'negative' used in a scientific sense is exactly that - a 'negative'. In colloquial usage, 'negative' means bad. The headline is just using this semantic difference as clickbait.

nonbel 11y ago
What happens is that direct replications are actively avoided so that the possibility of a negative (as in bad) citation does not come up. That appears to be the most popular "solution" to this problem, to the point it has been institutionalized and journals discourage publication of replications, etc. This effectively turns the area of research into pseudoscience in my mind. I don't see how a method that does not require independent replications can be considered science. I guess you could argue about the definition of science, but any method without replication is very different from the one that has brought great benefit to mankind.
- drvortex 11y ago
  >I don't see how a method that does not require independent replications can be considered science. There is an implicit benefit-of-doubt here - that the scientists themselves have independently replicated the experiment before publishing the result. Most papers will list how many times an experiment was repeated by providing the sample size, number of biological replicates, number of technical replicates, orthogonal evidence etc.
  - nonbel 11y ago
    >"the scientists themselves have independently replicated the experiment" Is this independent though? One person/group repeating the same experiment is much less convincing evidence of a stable phenomenon and control of the experimental situation. I take independent replication to mean others getting similar results. Going further, the ideal situation is when there is healthy rivalry (such as between universities), so the multiple research groups have incentive to find flaws with the claims of the others.
    - huac 11y ago
      The dynamic there is that there's little incentive to replicate results - no journal is going to publish an article based on that. Why spend your time and money on that then?
      - nonbel 11y ago
        >"Why spend your time and money on that then?" Well huac, That's a problem for those of us who think independent replication is a crucial part of the scientific method. If no one is going to attempt replicating my research and there are substantial obstacles placed in the way of me replicating other work... why waste my time? It doesn't pay very well, and it seems I am forced into a position of contributing to pseudoscience. Eventually only people who do not consider independent replication crucial will remain.
        huac 11y ago
        i don't disagree that independent replication is useful or crucial. the obstacle is one that is intrinsic to academia - 'publish or perish.' it is logical for a researcher to devote their time and energy to research that will be published, and journals just don't devote space to independent verification efforts.
        drvortex 11y ago
        Independent replication does occur when someone furthers the research, usually as a preliminary test of current knowledge before diving into new experiments. I personally took great joy when what I had reported was used and confirmed in the preliminary experiments done by other labs for their further studies. But there is a deeper philosophical issue with your idea of the scientific method : Consider what is meant by independent replication. Does it mean that the experiment must be repeated from scratch with different material etc. such that it is an independent test from the previous attempt? Or does it mean that it has to be repeated by different people as well ? It raises a conundrum. If people are a factor that must be controlled for by this definition, no replication can ever occur since the exact individuals that performed the experiment will have changed by definition during this 'replication' attempt. The skill and knowledge of people is a factor that is notoriously difficult to objectively quantify. This often results in statements such as "X method/assay/procedure does not work well in our hands ". On the other hand , if people are not a factor, then replication of the experiment by the same group of people will also qualify as an independent replicate.
- semi-extrinsic 11y ago
  I guess there is a fundamental dichotomy here between presenting a "method" : "you can do X and Y to measure the value of Z" and presenting a "final result" : "the value of Z for system W is N". In the former case, people will absolutely replicate you, and if your method is bogus you'll be called out on it.

nonbel 11y ago

What happens is that direct replications are actively avoided so that the possibility of a negative (as in bad) citation does not come up. That appears to be the most popular "solution" to this problem, to the point it has been institutionalized and journals discourage publication of replications, etc. This effectively turns the area of research into pseudoscience in my mind. I don't see how a method that does not require independent replications can be considered science. I guess you could argue about the definition of science, but any method without replication is very different from the one that has brought great benefit to mankind.

drvortex 11y ago
>I don't see how a method that does not require independent replications can be considered science. There is an implicit benefit-of-doubt here - that the scientists themselves have independently replicated the experiment before publishing the result. Most papers will list how many times an experiment was repeated by providing the sample size, number of biological replicates, number of technical replicates, orthogonal evidence etc.
- nonbel 11y ago
  >"the scientists themselves have independently replicated the experiment" Is this independent though? One person/group repeating the same experiment is much less convincing evidence of a stable phenomenon and control of the experimental situation. I take independent replication to mean others getting similar results. Going further, the ideal situation is when there is healthy rivalry (such as between universities), so the multiple research groups have incentive to find flaws with the claims of the others.
  - huac 11y ago
    The dynamic there is that there's little incentive to replicate results - no journal is going to publish an article based on that. Why spend your time and money on that then?
    - nonbel 11y ago
      >"Why spend your time and money on that then?" Well huac, That's a problem for those of us who think independent replication is a crucial part of the scientific method. If no one is going to attempt replicating my research and there are substantial obstacles placed in the way of me replicating other work... why waste my time? It doesn't pay very well, and it seems I am forced into a position of contributing to pseudoscience. Eventually only people who do not consider independent replication crucial will remain.
      - huac 11y ago
        i don't disagree that independent replication is useful or crucial. the obstacle is one that is intrinsic to academia - 'publish or perish.' it is logical for a researcher to devote their time and energy to research that will be published, and journals just don't devote space to independent verification efforts.
      - drvortex 11y ago
        Independent replication does occur when someone furthers the research, usually as a preliminary test of current knowledge before diving into new experiments. I personally took great joy when what I had reported was used and confirmed in the preliminary experiments done by other labs for their further studies. But there is a deeper philosophical issue with your idea of the scientific method : Consider what is meant by independent replication. Does it mean that the experiment must be repeated from scratch with different material etc. such that it is an independent test from the previous attempt? Or does it mean that it has to be repeated by different people as well ? It raises a conundrum. If people are a factor that must be controlled for by this definition, no replication can ever occur since the exact individuals that performed the experiment will have changed by definition during this 'replication' attempt. The skill and knowledge of people is a factor that is notoriously difficult to objectively quantify. This often results in statements such as "X method/assay/procedure does not work well in our hands ". On the other hand , if people are not a factor, then replication of the experiment by the same group of people will also qualify as an independent replicate.
semi-extrinsic 11y ago
I guess there is a fundamental dichotomy here between presenting a "method" : "you can do X and Y to measure the value of Z" and presenting a "final result" : "the value of Z for system W is N". In the former case, people will absolutely replicate you, and if your method is bogus you'll be called out on it.