Peer review is something I've been thinking about a lot lately as an outsider. I've never gone through a formal peer review process or worked in academia, but i

Peer review is something I've been thinking about a lot lately as an outsider. I've never gone through a formal peer review process or worked in academia, but it's pretty obvious that the process has significant problems. However, the problem that peer reviews are trying to solve (conferring legitimacy on a paper by the scientific community) is so important that we can't give up on it. Sure, pre-print servers mean science can move faster, but it also means anything can be "science." If we normalize publishing papers as preprints, there's very little stopping conspiracy theorists from publishing a paper and having it look plausible. The sudden rush of media coverage for COVID research has really exasperated the problem. Journalists should responsibly cover preprint studies, but frankly most don't, and who can blame them when even the scientific community is struggling to review the papers itself. I volunteer with a group[0] that's trying to address this problem in a small way by creating visual summaries/reviews of popular preprints for mainstream readers. This is a side project, so the review process is just a shared Google doc for each paper, but what I'd like to see: As a reviewer: * View diffs between paper revisions * Comment on papers inline, like a pull request * +1/-1 a submission, or just leave questions without a decision * Gain "karma" for good reviews to build my reputation in the community (like Stack OverFlow) As a paper author: * Reply to comments to understand and explain context * Post revisions As a reader: * See visual summaries of papers, with inlined reviews (like Research Explained) * Vote on both studies and reviews to surface the good ones * Browse profiles of authors and reviewers to understand their qualifications/contributions I do think https://distill.pub https://distill.pub is on the right track. Their papers are significantly more visual/readable than most, and they require contributors to open a pull request against the distill repo. However, their process isn't approachable for non-technical fields. [0]: https://www.researchexplained.org https://www.researchexplained.org