5 ms·

This is amazing. The trust of peer review as a stamp of quality among academics is dwindling, and distrust of science among the population is growing (within th

This is amazing. The trust of peer review as a stamp of quality among academics is dwindling, and distrust of science among the population is growing (within the increasing politicisation of some areas of research). Transparent peer review raises the bar for both academics and enhances the potential trust in the process. This is desperately needed as AI could further deteriorate the quality of science if the publishing process is not made more strict. This represents a significant step forward in rigorous science. I hope other publications follow suit.

thfuran 1y ago
>and enhances the potential trust in the process. I highly doubt it's a meaningful factor in public trust.
Ar-Curunir 1y ago
The general public does not distrust science because peer reviews are not public.
- kergonath 1y ago
  Right. The general public cannot understand the points that are being made during peer review.
jfengel 1y ago
Of the people who distrust science, how many of them have ever read a scientific paper? I suspect the number is low. If that's the case, they're unlikely to be more convinced by the presence of published peer review, either.
- oerpli 1y ago
  I would go further: Anyone who has published a "scientific paper" in the last decade or so either "distrusts science" or is more likely than not a mid-wit at best. Your posting doesn't give me the impression that you're very familiar with "science".
  - iorrus 1y ago
    Absolutely, the quickest way to lose faith in "The Science" is actually to do Science in a formal research institute....
    - petschge 1y ago
      Seeing how the sausage gets made makes you realize ALL the downsides and things you'd rather not have known. That doesn't mean you can think of a way to fix science. Let alone get the required funding to make an actual attempt.
    - probably_wrong 1y ago
      Isn't that true for everything, though? If I weren't a software developer I wouldn't know that I have to worry about questions like "has this plane been rebooted in the last 51 days?" [1] or "does this bank offer anything other than SMS as second factor?". Maybe structural engineers feel safer after their Master's when they traverse a bridge, but I bet that's more the exception than the rule. [1] https://www.theregister.com/2020/04/02/boeing_787_power_cycle_51_days_stale_data/ https://www.theregister.com/2020/04/02/boeing_787_power_cycl...
  - kergonath 1y ago
    > Anyone who has published a "scientific paper" in the last decade or so either "distrusts science" or is more likely than not a mid-wit at best. That is very unhelpful, to say the least. The amount of noise has increased, but it does not mean that the scientists who know their subject disappeared. They are still around and not any less bright than their predecessors were 30 years ago.
    - rcxdude 1y ago
      Steel-manning their argument, 'distrusting science' doesn't mean they throw the whole thing out, they're just aware that there is disagreement and bullshit going around within the process. As far as I can tell, it's dangerous to try to assess a topic through reading papers alone. A scientist active in the field will have read more widely, be aware of the reputations and biases of the different groups, and likely will have tried some of the published experiments themselves (replication does happen, if it's an interesting result, it's just rarely worth publishing the result).
- NoMoreNicksLeft 1y ago
  >Of the people who distrust science, Why should anyone trust science? Skepticism should always be the default position. Putting it on a pedestal to be worshipped is what led us all into this mess. If science needs trust to work, then whatever it is doing is something I'd like to see fail.
  - JumpCrisscross 1y ago
    > Why should anyone trust science? When people talk about those who distrust science, they aren't referring to the carefully sceptical. They're talking about people who come to a conclusion and then reject any evidence against it.
    - mikeyouse 1y ago
      Right. RFK supporters aren’t carefully weighing evidence to arrive at their conclusions, they are blindly following someone who is a mindless contrarian with a 30yr vendetta against vaccines. No amount or quality of peer review is going to change any of their minds.
      - JumpCrisscross 1y ago
        > RFK supporters aren’t carefully weighing evidence to arrive at their conclusions, they are blindly following someone I'd actually be curious what fraction of his supporters share his views on vaccines. It strikes me as more likely that they like one of his random, idiosyncratic views and are willing to excuse the anti-vaccine nonsense to get those policy outcomes.
    - fc417fc802 1y ago
      That, but also people who take the blanket view that none of the work produced by the scientific establishment is trustworthy. An ironic stance considering that it's the inverse of the blind worship that they accuse others of.
      - kergonath 1y ago
- aDyslecticCrow 1y ago
  I think there are a few groups and reasons of distrust. Some more or less valid. Those that distrust authority as a whole and lean into conspiracies cannot be saved with this kind of thing. But i think news and science are having similar perception issues recently. Distrust for news growing among the average population (for some good reasons). People are loosing faith in the objectivity of established media organisations. Most people are exposed to science through these traditional news. So adding back some sense of confidence and authority to scientific institutions is very valuable to non-academics. Even if they themselves would not read the papers or revews.
- Dig1t 1y ago
  The fact remains that distrust of science continues to grow. Up to now the establishment’s response has been one of condescension. Your comment echoes that attitude. Ignoring the problem is not going to fix it, and in fact continuing to regard these people as beneath you is only going to accelerate the downfall of this system.
  - vkou 1y ago
    > The fact remains that distrust of science continues to grow. Of course it does, billion-dollar interests who have a vested interest in attacking it have a well-funded propaganda arm, and as we've been discovering over the past century - angry, loud, and frequently repeated bullshit with a profit motive floods any signal out of the room. What's really sad is people who have legitimate concerns or desires for improvement hitching their horse-cart to the former. No amount of peer review or replication is going to convince someone whose fortune is built on peddling snake oil. > Ignoring the problem is not going to fix it, Here's a solution. Hold the people making these attacks to the same level of rigor as what they are attacking. Stop giving proven liars a loudspeaker. The day the press will do that is the day some meaningful progress to fix that damage may be made. Until you do that, we're going to continue getting shit outcomes.
- mike_hearn 1y ago
  Lots of them have. Look at any site where science skepticism is regularly posted, you will find that maybe a good 50% of the content is commentary on papers. Of those who don't trust science and haven't read papers, they will usually have read commentary by those who have. Source: I've published such skepticism in the past and met people who read my articles, including politicians. Nonetheless, you're right that simply publishing peer reviews won't help improve trust. The situation is bad enough that there's no One Weird Trick that can dig academia out of its hole. Some of the most intellectually fraudulent papers I've written about in the past did have transparent peer review, and all it showed was that peer reviewers were often aware of the critical problems found inside and waved it through anyway. Or that their feedback was ignored. Or that they agreed with obviously bogus practices. Or that the parts of the paper that revealed the problems weren't reviewed (eg. appendices, github repositories). After all, nobody cares about papers that got rejected by the system, the distrust is driven by the papers that weren't, so almost by definition such papers either were badly reviewed or the review wasn't used.
- deleted 1y ago
  [deleted]
kergonath 1y ago
> The trust of peer review as a stamp of quality among academics is dwindling The thing is, peer review is not a stamp of quality, and never was. It is just the basic level of due diligence. The referees cannot reproduce the results most of the time for a lot of very good reasons. They are here as a sanity check, to ensure that the work avoided common pitfalls and actually makes sense. What most people do not understand is that articles are not good because they are peer-reviewed; it’s the lack of peer review that is a red flag. Amongst reviewed articles, a lot of them will turn out to be wrong or flawed in ways that are impossible for the reviewers to find out.

thfuran 1y ago

>and enhances the potential trust in the process. I highly doubt it's a meaningful factor in public trust.

Ar-Curunir 1y ago

The general public does not distrust science because peer reviews are not public.

kergonath 1y ago
Right. The general public cannot understand the points that are being made during peer review.

jfengel 1y ago

Of the people who distrust science, how many of them have ever read a scientific paper? I suspect the number is low. If that's the case, they're unlikely to be more convinced by the presence of published peer review, either.

oerpli 1y ago
I would go further: Anyone who has published a "scientific paper" in the last decade or so either "distrusts science" or is more likely than not a mid-wit at best. Your posting doesn't give me the impression that you're very familiar with "science".
- iorrus 1y ago
  Absolutely, the quickest way to lose faith in "The Science" is actually to do Science in a formal research institute....
  - petschge 1y ago
    Seeing how the sausage gets made makes you realize ALL the downsides and things you'd rather not have known. That doesn't mean you can think of a way to fix science. Let alone get the required funding to make an actual attempt.
  - probably_wrong 1y ago
    Isn't that true for everything, though? If I weren't a software developer I wouldn't know that I have to worry about questions like "has this plane been rebooted in the last 51 days?" [1] or "does this bank offer anything other than SMS as second factor?". Maybe structural engineers feel safer after their Master's when they traverse a bridge, but I bet that's more the exception than the rule. [1] https://www.theregister.com/2020/04/02/boeing_787_power_cycle_51_days_stale_data/ https://www.theregister.com/2020/04/02/boeing_787_power_cycl...
- kergonath 1y ago
  > Anyone who has published a "scientific paper" in the last decade or so either "distrusts science" or is more likely than not a mid-wit at best. That is very unhelpful, to say the least. The amount of noise has increased, but it does not mean that the scientists who know their subject disappeared. They are still around and not any less bright than their predecessors were 30 years ago.
  - rcxdude 1y ago
    Steel-manning their argument, 'distrusting science' doesn't mean they throw the whole thing out, they're just aware that there is disagreement and bullshit going around within the process. As far as I can tell, it's dangerous to try to assess a topic through reading papers alone. A scientist active in the field will have read more widely, be aware of the reputations and biases of the different groups, and likely will have tried some of the published experiments themselves (replication does happen, if it's an interesting result, it's just rarely worth publishing the result).
NoMoreNicksLeft 1y ago
>Of the people who distrust science, Why should anyone trust science? Skepticism should always be the default position. Putting it on a pedestal to be worshipped is what led us all into this mess. If science needs trust to work, then whatever it is doing is something I'd like to see fail.
- JumpCrisscross 1y ago
  > Why should anyone trust science? When people talk about those who distrust science, they aren't referring to the carefully sceptical. They're talking about people who come to a conclusion and then reject any evidence against it.
  - mikeyouse 1y ago
    Right. RFK supporters aren’t carefully weighing evidence to arrive at their conclusions, they are blindly following someone who is a mindless contrarian with a 30yr vendetta against vaccines. No amount or quality of peer review is going to change any of their minds.
    - JumpCrisscross 1y ago
      > RFK supporters aren’t carefully weighing evidence to arrive at their conclusions, they are blindly following someone I'd actually be curious what fraction of his supporters share his views on vaccines. It strikes me as more likely that they like one of his random, idiosyncratic views and are willing to excuse the anti-vaccine nonsense to get those policy outcomes.
  - fc417fc802 1y ago
    That, but also people who take the blanket view that none of the work produced by the scientific establishment is trustworthy. An ironic stance considering that it's the inverse of the blind worship that they accuse others of.
    - kergonath 1y ago
aDyslecticCrow 1y ago
I think there are a few groups and reasons of distrust. Some more or less valid. Those that distrust authority as a whole and lean into conspiracies cannot be saved with this kind of thing. But i think news and science are having similar perception issues recently. Distrust for news growing among the average population (for some good reasons). People are loosing faith in the objectivity of established media organisations. Most people are exposed to science through these traditional news. So adding back some sense of confidence and authority to scientific institutions is very valuable to non-academics. Even if they themselves would not read the papers or revews.
Dig1t 1y ago
The fact remains that distrust of science continues to grow. Up to now the establishment’s response has been one of condescension. Your comment echoes that attitude. Ignoring the problem is not going to fix it, and in fact continuing to regard these people as beneath you is only going to accelerate the downfall of this system.
- vkou 1y ago
  > The fact remains that distrust of science continues to grow. Of course it does, billion-dollar interests who have a vested interest in attacking it have a well-funded propaganda arm, and as we've been discovering over the past century - angry, loud, and frequently repeated bullshit with a profit motive floods any signal out of the room. What's really sad is people who have legitimate concerns or desires for improvement hitching their horse-cart to the former. No amount of peer review or replication is going to convince someone whose fortune is built on peddling snake oil. > Ignoring the problem is not going to fix it, Here's a solution. Hold the people making these attacks to the same level of rigor as what they are attacking. Stop giving proven liars a loudspeaker. The day the press will do that is the day some meaningful progress to fix that damage may be made. Until you do that, we're going to continue getting shit outcomes.
mike_hearn 1y ago
Lots of them have. Look at any site where science skepticism is regularly posted, you will find that maybe a good 50% of the content is commentary on papers. Of those who don't trust science and haven't read papers, they will usually have read commentary by those who have. Source: I've published such skepticism in the past and met people who read my articles, including politicians. Nonetheless, you're right that simply publishing peer reviews won't help improve trust. The situation is bad enough that there's no One Weird Trick that can dig academia out of its hole. Some of the most intellectually fraudulent papers I've written about in the past did have transparent peer review, and all it showed was that peer reviewers were often aware of the critical problems found inside and waved it through anyway. Or that their feedback was ignored. Or that they agreed with obviously bogus practices. Or that the parts of the paper that revealed the problems weren't reviewed (eg. appendices, github repositories). After all, nobody cares about papers that got rejected by the system, the distrust is driven by the papers that weren't, so almost by definition such papers either were badly reviewed or the review wasn't used.
deleted 1y ago
[deleted]

kergonath 1y ago

> The trust of peer review as a stamp of quality among academics is dwindling The thing is, peer review is not a stamp of quality, and never was. It is just the basic level of due diligence. The referees cannot reproduce the results most of the time for a lot of very good reasons. They are here as a sanity check, to ensure that the work avoided common pitfalls and actually makes sense. What most people do not understand is that articles are not good because they are peer-reviewed; it’s the lack of peer review that is a red flag. Amongst reviewed articles, a lot of them will turn out to be wrong or flawed in ways that are impossible for the reviewers to find out.