5 ms·

No. This belongs in a journal dedicated to the field of study, not a general issue journal like Science. The authors are silly to think otherwise. It is also n

No. This belongs in a journal dedicated to the field of study, not a general issue journal like Science. The authors are silly to think otherwise. It is also not surprising that Science published a flashy article without a lot of oomph behind it. It's what they do.

pdonis 7y ago
So a journal can get a free pass on an obvious violation of scientific ethics just by saying that they're a general issue journal? The field of study has nothing to do with it. If a journal publishes a result, they are ethically obligated to publish further work that refutes that result.
- dash2 7y ago
  “If a journal publishes a result, they are ethically obligated to publish further work that refutes that result.” Too general. Few in science would accept this. The claim here is the weaker one that journals are ethically obliged to accept failed replication studies - and even that is non-obvious to me. (Should Science also be obliged to accept _successful_ replications? No? So it should accept papers on the basis solely of the results? Hello, publication bias.)
  - vinay_ys 7y ago
    Aren't they obligated to investigate? If the methodology applied is sound and conclusions are different, isn't that scientific progress in and of itself? The ask here is to send it for peer-review so that they get some different points of view which may guide their own further study. They are ok with rejection but not without peer-review.
  - Hasknewbie 7y ago
    Nice attempt at reframing there, but if this new paper is correct then it's not a 'failed replication', it's a refutation. Big difference.
- londons_explore 7y ago
  A middle ground could exist here... Rather than publish the entire paper, they could publish the single sentence: > Issue 55, Page 8, the paper we published has been replicated with more data, and results do not support the original research conclusions. See paper entitled "Foo" published in "other journal" for details.
- Misdicorl 7y ago
  Refutations and competing null results get published in different journals all the time. Seminal results get shown to be kind of not seminal 10 years later more often than you might think. > If a journal publishes a result, they are ethically obligated to publish further work that refutes that result. This isn't a violation of scientific ethics and I think its a silly argument to make. Journals aren't the parents of research results they publish and somehow responsible for all counter arguments or competing data points.
justinclift 7y ago
Doesn't taking that approach then become something like: "Drinking goats milk gives X-ray vision𝆯, according to new study!" ... 𝆯 - Accuracy to be followed up in paragraph 42, subsection 3, in a subsequent journal from a third party. So, the flashy headline gets used by "whoever" wants clicks (etc), but the follow up details are unlikely to be seen. Sounds like a literal plan for knowingly spreading misinformation. Side note - Anyknow know how to escape an asterisk here on HN, so it displays instead of starting an italic block? Only workaround I've found is to use 𝆯 instead, which isn't quite right. :/
- Misdicorl 7y ago
  I mean, I guess its a mild problem? Its no different than pulling a quote from a pay to publish article though, the only difference is the amount of cachet these journals have. Science/Nature/etc are broadcast journals. They're for publishing results that people outside your field would find interesting. Typically these are new and flashy results and sometimes theyre slightly suspect. Mostly theyre not. A refutation or null result typically isn't interesting to the audience of these journals- nor should it be.
  - justinclift 7y ago
    > ... nor should it be. This bit I don't really understand. If someone is interested in X, why would they be ok with carrying around what turns out to (likely be) incorrect knowledge about it? That approach is more equated with (say) Wired, who are famously inaccurate in many details "for a good story". eg Wired is entertainment. That's not what Science (the journal) is supposed to be about though. Or is it?
    - Misdicorl 7y ago
      Yes, believing false knowledge is unsettling. But it's unlikely the results in question were that momentous that they stayed with people outside the field. If the results were that, impactful the refuting article would belong in Science.
tathougies 7y ago
> It's what they do. Indeed, legally and ethically Science is free to do what they want and should be held so by all external parties. However, we, its consumers, are equally free to no longer pay attention to it, since its quality as a scientific journal is somewhat suspect. Anyone who truly believes in science (with a small s), ought to appropriately lower the amount of prestige with which they view science after such accusations.
- Misdicorl 7y ago
  Well, its more like new and flashy results are likely because new techniques were developed or a technique was applied in a novel way. Science/Nature/etc are publishing the first results with these new techniques so of course theyre going to be a bit buggy. Subsequent refinements and discovery of systematics and how to account for them improve the research, and often change the results, but those results don't make it to Science/Nature/etc because they don't belong there (usually). So, really you should be taking the mindset that articles in these journals should be taken with a grain of salt. Whether that should have an impact on their prestige? I suppose thats a personal decision.

pdonis 7y ago

So a journal can get a free pass on an obvious violation of scientific ethics just by saying that they're a general issue journal? The field of study has nothing to do with it. If a journal publishes a result, they are ethically obligated to publish further work that refutes that result.

dash2 7y ago
“If a journal publishes a result, they are ethically obligated to publish further work that refutes that result.” Too general. Few in science would accept this. The claim here is the weaker one that journals are ethically obliged to accept failed replication studies - and even that is non-obvious to me. (Should Science also be obliged to accept _successful_ replications? No? So it should accept papers on the basis solely of the results? Hello, publication bias.)
- vinay_ys 7y ago
  Aren't they obligated to investigate? If the methodology applied is sound and conclusions are different, isn't that scientific progress in and of itself? The ask here is to send it for peer-review so that they get some different points of view which may guide their own further study. They are ok with rejection but not without peer-review.
- Hasknewbie 7y ago
  Nice attempt at reframing there, but if this new paper is correct then it's not a 'failed replication', it's a refutation. Big difference.
londons_explore 7y ago
A middle ground could exist here... Rather than publish the entire paper, they could publish the single sentence: > Issue 55, Page 8, the paper we published has been replicated with more data, and results do not support the original research conclusions. See paper entitled "Foo" published in "other journal" for details.
Misdicorl 7y ago
Refutations and competing null results get published in different journals all the time. Seminal results get shown to be kind of not seminal 10 years later more often than you might think. > If a journal publishes a result, they are ethically obligated to publish further work that refutes that result. This isn't a violation of scientific ethics and I think its a silly argument to make. Journals aren't the parents of research results they publish and somehow responsible for all counter arguments or competing data points.

justinclift 7y ago

Doesn't taking that approach then become something like: "Drinking goats milk gives X-ray vision𝆯, according to new study!" ... 𝆯 - Accuracy to be followed up in paragraph 42, subsection 3, in a subsequent journal from a third party. So, the flashy headline gets used by "whoever" wants clicks (etc), but the follow up details are unlikely to be seen. Sounds like a literal plan for knowingly spreading misinformation. Side note - Anyknow know how to escape an asterisk here on HN, so it displays instead of starting an italic block? Only workaround I've found is to use 𝆯 instead, which isn't quite right. :/

Misdicorl 7y ago
I mean, I guess its a mild problem? Its no different than pulling a quote from a pay to publish article though, the only difference is the amount of cachet these journals have. Science/Nature/etc are broadcast journals. They're for publishing results that people outside your field would find interesting. Typically these are new and flashy results and sometimes theyre slightly suspect. Mostly theyre not. A refutation or null result typically isn't interesting to the audience of these journals- nor should it be.
- justinclift 7y ago
  > ... nor should it be. This bit I don't really understand. If someone is interested in X, why would they be ok with carrying around what turns out to (likely be) incorrect knowledge about it? That approach is more equated with (say) Wired, who are famously inaccurate in many details "for a good story". eg Wired is entertainment. That's not what Science (the journal) is supposed to be about though. Or is it?
  - Misdicorl 7y ago
    Yes, believing false knowledge is unsettling. But it's unlikely the results in question were that momentous that they stayed with people outside the field. If the results were that, impactful the refuting article would belong in Science.

tathougies 7y ago

> It's what they do. Indeed, legally and ethically Science is free to do what they want and should be held so by all external parties. However, we, its consumers, are equally free to no longer pay attention to it, since its quality as a scientific journal is somewhat suspect. Anyone who truly believes in science (with a small s), ought to appropriately lower the amount of prestige with which they view science after such accusations.

Misdicorl 7y ago
Well, its more like new and flashy results are likely because new techniques were developed or a technique was applied in a novel way. Science/Nature/etc are publishing the first results with these new techniques so of course theyre going to be a bit buggy. Subsequent refinements and discovery of systematics and how to account for them improve the research, and often change the results, but those results don't make it to Science/Nature/etc because they don't belong there (usually). So, really you should be taking the mindset that articles in these journals should be taken with a grain of salt. Whether that should have an impact on their prestige? I suppose thats a personal decision.