8 ms·

I'm going to be honest. IMHO publishing a paper should be like doing a PR on github and there is no reason it shouldn't be. You do your research, read the "PR"

I'm going to be honest. IMHO publishing a paper should be like doing a PR on github and there is no reason it shouldn't be. You do your research, read the "PR" i.e. submission guidelines and submit it, - sure let the committer and reviewers be anonymized until the PR has been accepted - then just pull it into a "Repo" aka journal and be done. How can this process cost tons of money?

philipodonnell 9y ago
How would you incentivize peer review with your approach?
- codelord 9y ago
  Do you think Springer pays for reviewing? No publisher does. It's all voluntary work by the same people who write the papers. It's the modern day slavery.
  - philipodonnell 9y ago
    The parent comment implied that reviewers would be anonymized with this Github-based approach unless the PR was approved, which seems like it would lead to more acceptances so that the reviewers could get credit.
- jfaucett 9y ago
  First, I'd question that you really need to incentivize it. People - myself included - are doing research in a topic because they are passionate about it and most are more than happy to look at new research and ideas about things they are passionate about. Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Let's say however that you need incentivization, I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information.
  - philipodonnell 9y ago
    > Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Perhaps not. But people keep telling me in this thread that reviewers work because the journals expect them to or they will get future papers rejected/lose conference access, not for the love of the topic or because they get paid or otherwise any kind of special visibility for it. I wonder whether an approach that promised only publicity for having done reviews or relying on altruism would really produce the quality of reviews necessary. > I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information. That's not really incentivizing the _reviewers_, though, right?
- cat199 9y ago
  not parent, but research association conference presentations might help to centralize announcements/review efforts but also keep things under researcher purview..
- sevensor 9y ago
  The same way we do now? There's a strong cultural expectation that one will agree to review, or have one's graduate students review, a reasonable number of papers. If you're a member of the Foo committee of the Society for Bar, you're not going to refuse to pull your weight as a reviewer for the Society for Bar's International Journal of Foo unless you want to get the cold shoulder at your next conference. It's not like anybody gets paid for this today.
- chr1 9y ago
  By providing opportunity to find errors in research of people you don't like! More seriously though, peer review is usually free or almost free, reviewers do not get anything near to the ridiculous amount of the money that "open access" journals demand. Besides, having articles on github, with all the data, so that any reader can check the results, and comment, would be much more powerful than the type of peer review we have today.
- deleted 9y ago
  [deleted]
- RangerScience 9y ago
  Clearly, blockchain! But no seriously, this seems like an actual technology / problem fit, although you'd need a decent amount of tooling, product design, etc for it to work - but some of that's already being explored by the "open source + blockchain" endeavors. In basic form, I imagine you'd basically turn the transaction fees into review fees. (The product design and tooling comes from how you prevent abuses to that basic format, I think)
  - philipodonnell 9y ago
    I actually think this is a pretty good use case, since it fits the decentralized trust aspect that many common blockchain use cases lack. > some of that's already being explored by the "open source + blockchain" endeavors. Any links?
    - fnordsensei 9y ago
      I work for an Open Access publisher/research discovery technology company, and we literally have this on the drawing board. We're small, but well connected in the world of publishing and research. I for one would be also be interested in links. One thing we don't want to do is reinvent the actual blockchain technology, so we're also interested in finding partnerships in terms of platform technology.
      - philipodonnell 9y ago
        A proof of stake smart contract would be perfect for this. The currency is reputation earned by writing well-done reviews and 'staked' behind reviews they believe are well-done. Papers need a certain number of well-done reviews with the stakes on both sides accumulating behind approve/deny. You have to pay with the currency to have your paper reviewed and the currencies are distributed to the reviewers to allow monetization via exchanges with a small amount withdrawn as a siphon to support the continued development of the network and marketing/public relations.
        fnordsensei 9y ago
- jononor 9y ago
  Reviewers get their names on the paper too. Which means if the paper was bad, well, they can be held responsible for it too.
  - roel_v 9y ago
    In what field is that? I've never seen this happen, nor would I want to be on any paper I merely reviewed.
    - jokla 9y ago
      I think this was meant as a suggestion, I've also never seen this happen. However, I am wondering why and so may I ask why you do not want to be mentioned as reviewer on a paper, maybe even with your review?
fnordsensei 9y ago
As a researcher, in a lot of cases, you want to be published in a reputable journal. Publishers have learned to charge for the reputation over the years. Reputation has a lot of inertia. The PR model would have to answer the question of how reputation is built. A "star" system would likely be inadequate for a number of reasons. But the major obstacle would be in getting researchers on board with the new model. Early adopters would have to withstand scoffs from their peers who haven't accepted the new model yet. Even Open Access, which could be regarded as an incremental improvement, has years, if not decades, of grueling work behind it. The network effect is very strong within the research community, which results in a situation which is very close to "all or nothing."
- jjoonathan 9y ago
  > Early adopters would have to withstand scoffs from their peers More importantly: punishment from the people who give them money.
- PopePompous 9y ago
  Also, the persons most comfortable with a radically different publishing model like PR are apt to be young researchers, but they are the ones most in need of establishing a reputation by publishing in established, prestigious journals.
  - fnordsensei 9y ago
    Spot on. There's a huge potential for optimization in the entire field, but also an amount of inertia rarely seen elsewhere.
- zentiggr 9y ago
  Sounds like a case for "shaming" those who rely on 'prestige' since that seems like the only way to break entrenched habits. I am not in research or publishing or anything else directly involved. This is just a generalization that exposure and denigration of egregious cases tend to be the most effective initiators of systemic change, given our hard-wired approval seeking.
ocschwar 9y ago
For the same reason that any journal with a full time editor and not a lot of subscribers costs a lot of money per issue or per subscription: The editor has to put in the hours to reject the crap that people submit, and those hours don't come free.
- jfaucett 9y ago
  > The editor has to put in the hours to reject the crap that people submit Does that process need to be any different from PRs on github? Can't all that be distributed across a team of researchers working on the topic of the journal, just as PRs on github are distributed across all developers working on the software?
  - jasode 9y ago
    >Does that process need to be any different from PRs on github? Can't all that be distributed across a team of researchers working on the topic of the journal, just as PRs on github It's not the same as a github PR because wading through a mountain of badly written papers is work researchers do not want to do. That's what a publisher (like Elsevier) does. See example of Elsevier employee Angelica Kerr.[1] It should be easy to see that scientists would think it's a waste of their time to do Angelica Kerr's work. I didn't get a chance to respond to the reply by allenz that suggested that journal chiefs should hire their own editing and administration staff. Again, that "solution" makes the same mistake of thinking scientists will do something they have no interest in doing. They don't want the hassles of "contracting" an editing team. The research publishing is not a "software problem" that's solvable by a "software platform solution." You can't solve it by applying social platform mechanics such as github PRs and/or votes of StackOverflow/Reddit. (My parent comment tries to explain this.[2]) [1] https://news.ycombinator.com/item?id=15278621 https://news.ycombinator.com/item?id=15278621 [2] https://news.ycombinator.com/item?id=15269673 https://news.ycombinator.com/item?id=15269673
    - jfaucett 9y ago
      So I read your links. How is Angela Kerr's job any different than a spam filter function? We've got production grade techniques for handling that now, and without going into technical details everything you mentioned as being in her set of tasks could be implemented using ML and NLP techniques, most of them not even that cutting edge. In many ways, an algorithmic approach could do her job better because some "crackpot" articles will not be crackpot but genius and if you give researchers the ability to tweak the filter settings by adjusting sensitivity / specificity to their own tastes you're mathematically guaranteed to increase the likelihood of getting the next breakthrough out there. The actual problem, which I think you implicitly identify when talking about Nature and Cell, is the "prestige" factor. As long as researchers are motivated by a prestige level of a journal b/c they may fear being ostracized by their peers or not getting recognition for their research - which has monetary and career costs involved - I think it will be very difficult to convince anyone to switch, regardless of how effective the platform could be. I haven't thought about that problem before so I'm not sure how to address it as of now.
dmitriz 9y ago
The useful part of the service does not a lot as many examples of community run journals shows, such as http://www.mathoa.org/journal-of-algebraic-combinatorics-flips-to-fair-oa/ http://www.mathoa.org/journal-of-algebraic-combinatorics-fli... The reason the publishers charge their exorbitant rates is because they have acquired the journals leaving no freedom to switch to alternative providers offering better service at lower cost. Nor do they allow saving costs by not using services that are not needed.

philipodonnell 9y ago

How would you incentivize peer review with your approach?

codelord 9y ago
Do you think Springer pays for reviewing? No publisher does. It's all voluntary work by the same people who write the papers. It's the modern day slavery.
- philipodonnell 9y ago
  The parent comment implied that reviewers would be anonymized with this Github-based approach unless the PR was approved, which seems like it would lead to more acceptances so that the reviewers could get credit.
jfaucett 9y ago
First, I'd question that you really need to incentivize it. People - myself included - are doing research in a topic because they are passionate about it and most are more than happy to look at new research and ideas about things they are passionate about. Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Let's say however that you need incentivization, I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information.
- philipodonnell 9y ago
  > Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Perhaps not. But people keep telling me in this thread that reviewers work because the journals expect them to or they will get future papers rejected/lose conference access, not for the love of the topic or because they get paid or otherwise any kind of special visibility for it. I wonder whether an approach that promised only publicity for having done reviews or relying on altruism would really produce the quality of reviews necessary. > I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information. That's not really incentivizing the _reviewers_, though, right?
cat199 9y ago
not parent, but research association conference presentations might help to centralize announcements/review efforts but also keep things under researcher purview..
sevensor 9y ago
The same way we do now? There's a strong cultural expectation that one will agree to review, or have one's graduate students review, a reasonable number of papers. If you're a member of the Foo committee of the Society for Bar, you're not going to refuse to pull your weight as a reviewer for the Society for Bar's International Journal of Foo unless you want to get the cold shoulder at your next conference. It's not like anybody gets paid for this today.
chr1 9y ago
By providing opportunity to find errors in research of people you don't like! More seriously though, peer review is usually free or almost free, reviewers do not get anything near to the ridiculous amount of the money that "open access" journals demand. Besides, having articles on github, with all the data, so that any reader can check the results, and comment, would be much more powerful than the type of peer review we have today.
deleted 9y ago
[deleted]
RangerScience 9y ago
Clearly, blockchain! But no seriously, this seems like an actual technology / problem fit, although you'd need a decent amount of tooling, product design, etc for it to work - but some of that's already being explored by the "open source + blockchain" endeavors. In basic form, I imagine you'd basically turn the transaction fees into review fees. (The product design and tooling comes from how you prevent abuses to that basic format, I think)
- philipodonnell 9y ago
  I actually think this is a pretty good use case, since it fits the decentralized trust aspect that many common blockchain use cases lack. > some of that's already being explored by the "open source + blockchain" endeavors. Any links?
  - fnordsensei 9y ago
    I work for an Open Access publisher/research discovery technology company, and we literally have this on the drawing board. We're small, but well connected in the world of publishing and research. I for one would be also be interested in links. One thing we don't want to do is reinvent the actual blockchain technology, so we're also interested in finding partnerships in terms of platform technology.
    - philipodonnell 9y ago
      A proof of stake smart contract would be perfect for this. The currency is reputation earned by writing well-done reviews and 'staked' behind reviews they believe are well-done. Papers need a certain number of well-done reviews with the stakes on both sides accumulating behind approve/deny. You have to pay with the currency to have your paper reviewed and the currencies are distributed to the reviewers to allow monetization via exchanges with a small amount withdrawn as a siphon to support the continued development of the network and marketing/public relations.
      - fnordsensei 9y ago
jononor 9y ago
Reviewers get their names on the paper too. Which means if the paper was bad, well, they can be held responsible for it too.
- roel_v 9y ago
  In what field is that? I've never seen this happen, nor would I want to be on any paper I merely reviewed.
  - jokla 9y ago
    I think this was meant as a suggestion, I've also never seen this happen. However, I am wondering why and so may I ask why you do not want to be mentioned as reviewer on a paper, maybe even with your review?

fnordsensei 9y ago

As a researcher, in a lot of cases, you want to be published in a reputable journal. Publishers have learned to charge for the reputation over the years. Reputation has a lot of inertia. The PR model would have to answer the question of how reputation is built. A "star" system would likely be inadequate for a number of reasons. But the major obstacle would be in getting researchers on board with the new model. Early adopters would have to withstand scoffs from their peers who haven't accepted the new model yet. Even Open Access, which could be regarded as an incremental improvement, has years, if not decades, of grueling work behind it. The network effect is very strong within the research community, which results in a situation which is very close to "all or nothing."

jjoonathan 9y ago
> Early adopters would have to withstand scoffs from their peers More importantly: punishment from the people who give them money.
PopePompous 9y ago
Also, the persons most comfortable with a radically different publishing model like PR are apt to be young researchers, but they are the ones most in need of establishing a reputation by publishing in established, prestigious journals.
- fnordsensei 9y ago
  Spot on. There's a huge potential for optimization in the entire field, but also an amount of inertia rarely seen elsewhere.
zentiggr 9y ago
Sounds like a case for "shaming" those who rely on 'prestige' since that seems like the only way to break entrenched habits. I am not in research or publishing or anything else directly involved. This is just a generalization that exposure and denigration of egregious cases tend to be the most effective initiators of systemic change, given our hard-wired approval seeking.

ocschwar 9y ago

For the same reason that any journal with a full time editor and not a lot of subscribers costs a lot of money per issue or per subscription: The editor has to put in the hours to reject the crap that people submit, and those hours don't come free.

jfaucett 9y ago
> The editor has to put in the hours to reject the crap that people submit Does that process need to be any different from PRs on github? Can't all that be distributed across a team of researchers working on the topic of the journal, just as PRs on github are distributed across all developers working on the software?
- jasode 9y ago
  >Does that process need to be any different from PRs on github? Can't all that be distributed across a team of researchers working on the topic of the journal, just as PRs on github It's not the same as a github PR because wading through a mountain of badly written papers is work researchers do not want to do. That's what a publisher (like Elsevier) does. See example of Elsevier employee Angelica Kerr.[1] It should be easy to see that scientists would think it's a waste of their time to do Angelica Kerr's work. I didn't get a chance to respond to the reply by allenz that suggested that journal chiefs should hire their own editing and administration staff. Again, that "solution" makes the same mistake of thinking scientists will do something they have no interest in doing. They don't want the hassles of "contracting" an editing team. The research publishing is not a "software problem" that's solvable by a "software platform solution." You can't solve it by applying social platform mechanics such as github PRs and/or votes of StackOverflow/Reddit. (My parent comment tries to explain this.[2]) [1] https://news.ycombinator.com/item?id=15278621 https://news.ycombinator.com/item?id=15278621 [2] https://news.ycombinator.com/item?id=15269673 https://news.ycombinator.com/item?id=15269673
  - jfaucett 9y ago
    So I read your links. How is Angela Kerr's job any different than a spam filter function? We've got production grade techniques for handling that now, and without going into technical details everything you mentioned as being in her set of tasks could be implemented using ML and NLP techniques, most of them not even that cutting edge. In many ways, an algorithmic approach could do her job better because some "crackpot" articles will not be crackpot but genius and if you give researchers the ability to tweak the filter settings by adjusting sensitivity / specificity to their own tastes you're mathematically guaranteed to increase the likelihood of getting the next breakthrough out there. The actual problem, which I think you implicitly identify when talking about Nature and Cell, is the "prestige" factor. As long as researchers are motivated by a prestige level of a journal b/c they may fear being ostracized by their peers or not getting recognition for their research - which has monetary and career costs involved - I think it will be very difficult to convince anyone to switch, regardless of how effective the platform could be. I haven't thought about that problem before so I'm not sure how to address it as of now.

dmitriz 9y ago

The useful part of the service does not a lot as many examples of community run journals shows, such as http://www.mathoa.org/journal-of-algebraic-combinatorics-flips-to-fair-oa/ http://www.mathoa.org/journal-of-algebraic-combinatorics-fli... The reason the publishers charge their exorbitant rates is because they have acquired the journals leaving no freedom to switch to alternative providers offering better service at lower cost. Nor do they allow saving costs by not using services that are not needed.