5 ms·

The peer review process has reached a point where it's more harmful than helpful. What I want has following properties: * Version control, I can contribute to

The peer review process has reached a point where it's more harmful than helpful. What I want has following properties: * Version control, I can contribute to other people's papers * I can link to a sentence in another paper at some point in time so that when I read another paper, I can go to the location immediately * Open, like arxiv (anyone can publish anything), however arxiv discourages uploading personal or class projects, which is a terrible idea, the best part about GitHub is finding someone's abandoned project that does something * Full-text search I think that most academic publishing startups approach the problem by indexing already published papers. I think that starting clean slate is achievable.

Ar-Curunir 7y ago
None of the points you raised are about peer review...
- chr1 7y ago
  They are not about peer review, if by "peer review" you mean the abstract notion of someone reviewing your work. But peer review is a specific mechanism of getting comments about your work. And unfortunately it is a mechanism that has not been updated after invention of computers, so it is very inefficient at its job. Peer review in 21st century should be as easy and as open, as reviewing pull requests on github.
  - ISL 7y ago
    The comments are secondary. Filtering for error and importance are the primary goals of peer-review.
    - chr1 7y ago
      How does one "Filter for error and importance" without writing comments? I have been peer reviewer multiple times, and always had to write comments either to author pointing out errors and asking for clarifications or to the journal explaining why the paper is hopeless. It was not much different from reviewing pull requests by email. Did you have different experience reviewing papers for scientific journals?
      - ISL 7y ago
        For sure there are comments required to justify the decision/revisions. I was replying to the idea that one might submit an article for peer-review in order to get comments on one's work from the peer-review process, rather than to attempt to get it into a journal where it might be read by many people.
  - chr1 7y ago
    I am genuinely curious why these comments criticizing the current process of peer review and publishing are being downvoted. In my ten years of working in academia i have not met anyone who was happy with the current state of things. So i would be very interested to know are there so many scientist on hn who are happy with peer review process, are there many programmers who want to protect science, or are these comments terrible in some other way to deserve downvotes.
    - deleted 7y ago
      [deleted]
    - Ar-Curunir 7y ago
      It’s not that peer review is great (there’s lot of problems with it), but rather the fact that the points you brought up have nothing to do with peer review. Anyway, I agree that academia, and science in academia in particular, has lots of problems
Reedx 7y ago
It seems people are kneejerk downvoting you, but maybe the peer review process is a problem. It's worth questioning at least. Eric Weinstein (for one) has been talking about this a lot and argues that it could be better to work out errors in public, instead of defaulting to consensus. "The need for verification from others dampens the potential for imagination, which Weinstein believes to be essential for stretching beyond what is currently known to dream the unknowable. This, he argues, is how the sciences actually evolve. Weinstein points to a 1963 Scientific American article by Paul Dirac in which the theoretical physicist discusses the discovery of a third dimension, itself a revolutionary idea proposed by Newton, and then to four dimensions, as provided by Einstein. Imagination is required to make further theoretical leaps, which might require not listening to present-day consensus. Weinstein notes that when Crick and Watson published their seminal 1953 paper on the double helix, Nature did not need peer review to allow its publication. "It was an editor's job to figure out if it was worthy of publication." Thankfully, the editors allowed it; that paper revolutionized our understanding of molecular biology. Their work is the basis of all genetic research today." https://bigthink.com/technology-innovation/eric-weinstein-science https://bigthink.com/technology-innovation/eric-weinstein-sc...
- chr1 7y ago
  It's indeed strange to see the parent comment being downvoted. Especially by people who know what github is. Current academic journals are harmful for science, the same way as trying to use Roman numerals would be harmful for math. They are outdated means of communication that slows down progress. Science needs its github, and github could easily become that with very small effort.
  - adamnemecek 7y ago
    Github is too big now to adjust the course to follow this. They are going after enterprise. I think to pull this off, you need to be a "scientist outside of science". Also I'm not sure how this would make money.
fooker 7y ago
You are underestimating the amount of domain knowledge needed for this. Try contributing to open source software which is currently used for research, you'll realize the issue.
- adamnemecek 7y ago
  The domain knowledge being how academia works? My point is not to perfectly copy how academia works but to create a new platform that will allow people outside of academia to participate in the scientific process. Like maybe your skill is that you can fix up poorly written wording. This platform will allow you to specialize in one thing.
  - fooker 7y ago
    No, I meant domain knowledge to understand and contribute to most research.
ivalm 7y ago
This may work for some, but not many types of work. For example > * Version control [...] If I am working on a non-trivial system made of some novel-ish material doing some hard-ish measurements (I just described most of experimental physics/chemistry/material science) then there is approximately zero other people in the world who can reproduce without considerable effort. This reproduction effort is meaningful/justified once the original group finds some cool effect and ensures it is not spurious, but that's relatively late in the paper development (and at a point where it would make more sense to write a new paper for follow up rather than contribute to the current work).
- adamnemecek 7y ago
  Maybe they can fix spelling, maybe they can add graphics, or fix wording.

Ar-Curunir 7y ago

None of the points you raised are about peer review...

chr1 7y ago
They are not about peer review, if by "peer review" you mean the abstract notion of someone reviewing your work. But peer review is a specific mechanism of getting comments about your work. And unfortunately it is a mechanism that has not been updated after invention of computers, so it is very inefficient at its job. Peer review in 21st century should be as easy and as open, as reviewing pull requests on github.
- ISL 7y ago
  The comments are secondary. Filtering for error and importance are the primary goals of peer-review.
  - chr1 7y ago
    How does one "Filter for error and importance" without writing comments? I have been peer reviewer multiple times, and always had to write comments either to author pointing out errors and asking for clarifications or to the journal explaining why the paper is hopeless. It was not much different from reviewing pull requests by email. Did you have different experience reviewing papers for scientific journals?
    - ISL 7y ago
      For sure there are comments required to justify the decision/revisions. I was replying to the idea that one might submit an article for peer-review in order to get comments on one's work from the peer-review process, rather than to attempt to get it into a journal where it might be read by many people.
- chr1 7y ago
  I am genuinely curious why these comments criticizing the current process of peer review and publishing are being downvoted. In my ten years of working in academia i have not met anyone who was happy with the current state of things. So i would be very interested to know are there so many scientist on hn who are happy with peer review process, are there many programmers who want to protect science, or are these comments terrible in some other way to deserve downvotes.
  - deleted 7y ago
    [deleted]
  - Ar-Curunir 7y ago
    It’s not that peer review is great (there’s lot of problems with it), but rather the fact that the points you brought up have nothing to do with peer review. Anyway, I agree that academia, and science in academia in particular, has lots of problems

Reedx 7y ago

It seems people are kneejerk downvoting you, but maybe the peer review process is a problem. It's worth questioning at least. Eric Weinstein (for one) has been talking about this a lot and argues that it could be better to work out errors in public, instead of defaulting to consensus. "The need for verification from others dampens the potential for imagination, which Weinstein believes to be essential for stretching beyond what is currently known to dream the unknowable. This, he argues, is how the sciences actually evolve. Weinstein points to a 1963 Scientific American article by Paul Dirac in which the theoretical physicist discusses the discovery of a third dimension, itself a revolutionary idea proposed by Newton, and then to four dimensions, as provided by Einstein. Imagination is required to make further theoretical leaps, which might require not listening to present-day consensus. Weinstein notes that when Crick and Watson published their seminal 1953 paper on the double helix, Nature did not need peer review to allow its publication. "It was an editor's job to figure out if it was worthy of publication." Thankfully, the editors allowed it; that paper revolutionized our understanding of molecular biology. Their work is the basis of all genetic research today." https://bigthink.com/technology-innovation/eric-weinstein-science https://bigthink.com/technology-innovation/eric-weinstein-sc...

chr1 7y ago
It's indeed strange to see the parent comment being downvoted. Especially by people who know what github is. Current academic journals are harmful for science, the same way as trying to use Roman numerals would be harmful for math. They are outdated means of communication that slows down progress. Science needs its github, and github could easily become that with very small effort.
- adamnemecek 7y ago
  Github is too big now to adjust the course to follow this. They are going after enterprise. I think to pull this off, you need to be a "scientist outside of science". Also I'm not sure how this would make money.

fooker 7y ago

You are underestimating the amount of domain knowledge needed for this. Try contributing to open source software which is currently used for research, you'll realize the issue.

adamnemecek 7y ago
The domain knowledge being how academia works? My point is not to perfectly copy how academia works but to create a new platform that will allow people outside of academia to participate in the scientific process. Like maybe your skill is that you can fix up poorly written wording. This platform will allow you to specialize in one thing.
- fooker 7y ago
  No, I meant domain knowledge to understand and contribute to most research.

ivalm 7y ago

This may work for some, but not many types of work. For example > * Version control [...] If I am working on a non-trivial system made of some novel-ish material doing some hard-ish measurements (I just described most of experimental physics/chemistry/material science) then there is approximately zero other people in the world who can reproduce without considerable effort. This reproduction effort is meaningful/justified once the original group finds some cool effect and ensures it is not spurious, but that's relatively late in the paper development (and at a point where it would make more sense to write a new paper for follow up rather than contribute to the current work).

adamnemecek 7y ago
Maybe they can fix spelling, maybe they can add graphics, or fix wording.