6 ms·

I gave up reading The Lancet some time ago. They have a poor record frankly with this kind of thing, Andrew Wakefield et al chief among them. They should have b

by Gatsky 6y ago

I gave up reading The Lancet some time ago. They have a poor record frankly with this kind of thing, Andrew Wakefield et al chief among them. They should have been cancelled over that one. Anyway, the larger point is that these established journals have failed us. Their added value, already low, has dropped below zero in this pandemic. They are slow, opaque, rent seeking parasitic entities seemingly run by individuals who don’t grasp basic statistics. The combination of preprint servers + twitter has proven far more effective in looking after humanity’s best interests.

jxy 6y ago
It is exactly such kinds of ignorants of basic statistics, the faulty generalizations of the effectiveness of established journals, have failed the humanity.
- cracker_jacks 6y ago
  Is there a statistical demonstration of the effectiveness of established journals? It seems like you would have to do some form of counterfactual study to prove/disprove this. As far as I know, the argument of the effectiveness of journals only stems from the survivorship bias (e.g. all things considered research until recently has been published by journals).
  - SiempreViernes 6y ago
    No, you can look at the stuff people put up on the preprints to see what would happen if there's basically no screening at all.
    - cracker_jacks 6y ago
      The preprint model has actually been wildly successful for many communities (https://arxiv.org/ https://arxiv.org/, https://www.biorxiv.org/ https://www.biorxiv.org/). I don't think anyone is arguing against peer review, but the idea of peer review and an established journal is being conflated. If anything, publishing established journals can create a level of implied authority that reduces much need scrutiny.
Angostura 6y ago
> They have a poor record frankly with this kind of thing, Andrew Wakefield et al chief among them. Would you care to name some others?
- cgh 6y ago
  EAT-Lancet: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(19)31903-8/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6...
- asmithmd1 6y ago
  Autism and vaccine study https://www.npr.org/sections/health-shots/2010/02/lancet_wakefield_autism_mmr_au.html https://www.npr.org/sections/health-shots/2010/02/lancet_wak...
  - chimprich 6y ago
    That /is/ the Wakefield study.
- Gatsky 6y ago
  Of the top 10 most highly cited papers that have been retracted, 3 of them are in the Lancet, more than any other journal [1]. [1] https://retractionwatch.com/the-retraction-watch-leaderboard/top-10-most-highly-cited-retracted-papers/ https://retractionwatch.com/the-retraction-watch-leaderboard...
  - ufo 6y ago
    I'm not sure what kind of conclusion we can take from that dataset. It is only 10 papers and it doesn't take into account the average number of citations for the non-retracted articles in each journal.
    - Gatsky 6y ago
      I was asked to provide examples of the Lancet's poor track record, which I have done. I am not trying to prove it is better or worse than any other journal.
PiggySpeed 6y ago
No, you don't understand this. The Lancet is one of the most impactful journals in medicine, and has played an important role in shaping how medicine is practiced. It is rigorous in its process, but it's not foolproof. So yes, there are some bad studies that come through, but that is why healthcare professionals are TRAINED to interpret studies--we separate all the bullshit from the legitimately useful stuff. You should see some of the crap that comes through in some of the lower-impact journals. > The combination of preprint servers + twitter has proven far more effective in looking after humanity’s best interests. This view is incredibly false and dangerous for so many reasons.
- Gatsky 6y ago
  I am a healthcare professional, and I've published in one of the Lancet Journals. I understand perfectly fine. Here's the thing - The Lancet is not impactful. Researchers choose to publish their impactful research in The Lancet. Other researchers donate their time to peer review this research. It is the researchers that have shaped how medicine is practiced, not The Lancet. It is the researchers and the peer reviewers that are rigorous. The Lancet itself is none of these things, it is a business run by people who don't do research. Without the people who actually do the work, or the patients that volunteer for the research, it is nothing. The Lancet, like all top tier journals, has long forgotten this distinction. The problem isn't healthcare professionals lacking training. Take the Wakefield study. Did any doctor decide to stop offering vaccines because of that study? No. But it had a large impact on a the anti-vax narrative. These top journals have influence far beyond the professional sphere. This is why your suggestion we should give them a free pass is dangerous. I think it is clearly true that rapid publication and out in the open discussion and peer review is very healthy. I don't see how you could argue otherwise? Why wouldn't I want to be able to read the opinion of someone I respect intellectually eg Andrew Gelman, on a study they have decided to comment on? How is that 'dangerous' if I am a healthcare professional who should be able to critically review a published paper as you suggest? Why do I need to rely on the reviewers the journal has chosen? The Journals want to keep things as they are to maintain their importance and their bottom line. We suffer as a result.
  - DiogenesKynikos 6y ago
tobltobs 6y ago
> The combination of preprint servers + twitter Filtering noise with noise?
- jfk13 6y ago
  The great thing is that there's such a range of noise to choose from, you're sure to be able to find some you like.
jrowley 6y ago
I imagine it'd be hard to bring preprint studies + twitter threads to head of your department and use that as the basis for institutional policy change? Although it could change individual physician's practice which I guess is where things start?
sam36 6y ago
> Andrew Wakefield Is that really all it took? Here's 1000 more that came to similar conclusions: https://vaccine-injury.info/pdf/vaccinepeerreview.pdf https://vaccine-injury.info/pdf/vaccinepeerreview.pdf Have fun!
- Gatsky 6y ago
  Well there were people that thought books were the work of the devil when they first came out. It takes a 2 or 3 generations to get the most important ideas completely accepted. The Flynn effect is helpful here.
  - thu2111 6y ago
    The Wakefield argument is weaker than it should be. Wasn't his paper retracted because of its funding source, not because of specific errors in the paper itself? At least that's what Wikipedia's article on Wakefield claims. And of the co-authors, they asked for retraction because of the stink in the press over the funding, again, not because of actual errors in the science. Those don't actually seem like valid reasons for retraction, on their face. It had 12 co-authors so that's a lot of peer review to start with, before whatever the Lancet applied.
- BFatts 6y ago
  Uh, secret meetings and confidential cover-ups? This reads more like a spy novel than a trustworthy document. Sorry.

by Gatsky 6y ago

jxy 6y ago

It is exactly such kinds of ignorants of basic statistics, the faulty generalizations of the effectiveness of established journals, have failed the humanity.

cracker_jacks 6y ago
Is there a statistical demonstration of the effectiveness of established journals? It seems like you would have to do some form of counterfactual study to prove/disprove this. As far as I know, the argument of the effectiveness of journals only stems from the survivorship bias (e.g. all things considered research until recently has been published by journals).
- SiempreViernes 6y ago
  No, you can look at the stuff people put up on the preprints to see what would happen if there's basically no screening at all.
  - cracker_jacks 6y ago
    The preprint model has actually been wildly successful for many communities (https://arxiv.org/ https://arxiv.org/, https://www.biorxiv.org/ https://www.biorxiv.org/). I don't think anyone is arguing against peer review, but the idea of peer review and an established journal is being conflated. If anything, publishing established journals can create a level of implied authority that reduces much need scrutiny.

Angostura 6y ago

> They have a poor record frankly with this kind of thing, Andrew Wakefield et al chief among them. Would you care to name some others?

cgh 6y ago
EAT-Lancet: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(19)31903-8/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6...
asmithmd1 6y ago
Autism and vaccine study https://www.npr.org/sections/health-shots/2010/02/lancet_wakefield_autism_mmr_au.html https://www.npr.org/sections/health-shots/2010/02/lancet_wak...
- chimprich 6y ago
  That /is/ the Wakefield study.
Gatsky 6y ago
Of the top 10 most highly cited papers that have been retracted, 3 of them are in the Lancet, more than any other journal [1]. [1] https://retractionwatch.com/the-retraction-watch-leaderboard/top-10-most-highly-cited-retracted-papers/ https://retractionwatch.com/the-retraction-watch-leaderboard...
- ufo 6y ago
  I'm not sure what kind of conclusion we can take from that dataset. It is only 10 papers and it doesn't take into account the average number of citations for the non-retracted articles in each journal.
  - Gatsky 6y ago
    I was asked to provide examples of the Lancet's poor track record, which I have done. I am not trying to prove it is better or worse than any other journal.

PiggySpeed 6y ago

No, you don't understand this. The Lancet is one of the most impactful journals in medicine, and has played an important role in shaping how medicine is practiced. It is rigorous in its process, but it's not foolproof. So yes, there are some bad studies that come through, but that is why healthcare professionals are TRAINED to interpret studies--we separate all the bullshit from the legitimately useful stuff. You should see some of the crap that comes through in some of the lower-impact journals. > The combination of preprint servers + twitter has proven far more effective in looking after humanity’s best interests. This view is incredibly false and dangerous for so many reasons.

Gatsky 6y ago
I am a healthcare professional, and I've published in one of the Lancet Journals. I understand perfectly fine. Here's the thing - The Lancet is not impactful. Researchers choose to publish their impactful research in The Lancet. Other researchers donate their time to peer review this research. It is the researchers that have shaped how medicine is practiced, not The Lancet. It is the researchers and the peer reviewers that are rigorous. The Lancet itself is none of these things, it is a business run by people who don't do research. Without the people who actually do the work, or the patients that volunteer for the research, it is nothing. The Lancet, like all top tier journals, has long forgotten this distinction. The problem isn't healthcare professionals lacking training. Take the Wakefield study. Did any doctor decide to stop offering vaccines because of that study? No. But it had a large impact on a the anti-vax narrative. These top journals have influence far beyond the professional sphere. This is why your suggestion we should give them a free pass is dangerous. I think it is clearly true that rapid publication and out in the open discussion and peer review is very healthy. I don't see how you could argue otherwise? Why wouldn't I want to be able to read the opinion of someone I respect intellectually eg Andrew Gelman, on a study they have decided to comment on? How is that 'dangerous' if I am a healthcare professional who should be able to critically review a published paper as you suggest? Why do I need to rely on the reviewers the journal has chosen? The Journals want to keep things as they are to maintain their importance and their bottom line. We suffer as a result.
- DiogenesKynikos 6y ago

tobltobs 6y ago

> The combination of preprint servers + twitter Filtering noise with noise?

jfk13 6y ago
The great thing is that there's such a range of noise to choose from, you're sure to be able to find some you like.

jrowley 6y ago

I imagine it'd be hard to bring preprint studies + twitter threads to head of your department and use that as the basis for institutional policy change? Although it could change individual physician's practice which I guess is where things start?

sam36 6y ago

> Andrew Wakefield Is that really all it took? Here's 1000 more that came to similar conclusions: https://vaccine-injury.info/pdf/vaccinepeerreview.pdf https://vaccine-injury.info/pdf/vaccinepeerreview.pdf Have fun!

Gatsky 6y ago
Well there were people that thought books were the work of the devil when they first came out. It takes a 2 or 3 generations to get the most important ideas completely accepted. The Flynn effect is helpful here.
- thu2111 6y ago
  The Wakefield argument is weaker than it should be. Wasn't his paper retracted because of its funding source, not because of specific errors in the paper itself? At least that's what Wikipedia's article on Wakefield claims. And of the co-authors, they asked for retraction because of the stink in the press over the funding, again, not because of actual errors in the science. Those don't actually seem like valid reasons for retraction, on their face. It had 12 co-authors so that's a lot of peer review to start with, before whatever the Lancet applied.
BFatts 6y ago
Uh, secret meetings and confidential cover-ups? This reads more like a spy novel than a trustworthy document. Sorry.