7 ms·

I find it hard to believe the current system -- a secret scientific sanity check by a couple of people -- is the best we can do as far as gatekeeping in this da

by Dove 15y ago

I find it hard to believe the current system -- a secret scientific sanity check by a couple of people -- is the best we can do as far as gatekeeping in this day and age. I would much rather see original research managed OSS style -- published directly to communities of interest, critiqued and discussed openly, with revisions and explanations and discussions and raw data and source code and everything in all its ugly glory right there with it. There is a temptation under the current system to regard the contents of journals as absolute truth on account of their curation behind closed doors. We used to feel that way about newspapers, too. The truth is that credibility comes in degrees, and it's much safer to be able to see the guts of the operation.

adamjernst 15y ago
There's a thin line here. On one hand, you're right that openness is a good thing. On the other hand, modern science is incredibly political. When millions of dollars of funding is on the line, tensions run high. Anonymity is one way to shield reviewers. But the only way to have workable anonymous review is a trusted gatekeeper—and those are expensive.
lutorm 15y ago
I absolutely don't think that the current system means that the contents of journals are absolute truth -- I've been a referee and seen what happens to rejected articles (they show up a month later in another journal). The point is not so much about the quality of what does come out, it's about the quality of the crap that currently does not come out. While I agree that it could be useful to have open discussions about papers, I think if that happens at the expense of the ability of everyone who thinks they have an idea to put stuff out there, the result might be a net decrease in signal/noise.
jackpirate 15y ago
People propose that plan all the time. Here's why it can't work: While programmers like coding, researchers hate writing (they like doing research!). While there are relatively few OSS projects, there are millions of research papers (yes, millions! http://scholar.google.com/scholar?q=research&hl=en&btnG=Search&as_sdt=1%2C5 http://scholar.google.com/scholar?q=research&hl=en&b...) There are wiki style projects out there for each major domain. For example, quantum computing has http://www.quantiki.org http://www.quantiki.org. These sites are great, but they will never replace journal articles because they do not handle history or citations well. Imagine if all we had was wikipedia to tell us about the Church-Turing hypothesis? Papers like that give us an insight into the scientists' mind that nothing else can.
- forkandwait 15y ago
  It works now in physics: arxiv.org The reason it works is that your peers read your work, and either refute or cite you. Remember -- everybody knows everybody in a subfield, or at least everybody's advisor. It helps when there is math involved which is either correct or not. If somebody random submits something to arxiv, people know they don't know him or his grad school and that is thus probably a crackpot and so don't bother to read it (except sometimes someone from nowhere gets a result, which would NEVER be seen in the closed peer review process). EDIT: Also you attach your name to a finished product, not an unattributed ongoing wiki page. Your reputation is everything, so you win or lose based on the articles. One of the problems in social science academia is that 98% of the articles are completely worthless, and they only serve to get a professor promotion based on an index of citation times journal prestige. The smaller state colleges require their junior faculty to "publish", but they don't have enough to say to get into real journals and the publishing houses fill the need with "Journal of Rural Sociology" etc.
  - jackpirate 15y ago
    IANA Physicist, but I can't imagine any physics journals feel threatened by arxiv. When someone wins a Nobel prize for work that was published exclusively through arxiv, then you might have a point.
    - imrehg 15y ago
      I Am A Physicist, and researchers do use Arxiv quite often: pre-publication papers, or something that is written up and can iterated (unlike journals, Arxiv can store multiple version of the same paper). It is not peer reviewed but good for having feedback on things you are doing. Of course, publications from the "big guns" go straight to the big journals, and their peer review is more like rubber-stamp if you have the name on your author list. So the only difference is for the bureaucrats, who want you to show them "proper" papers to be able to move forward, the people doing the actual research only care whether your experiment and explanation is solid or not, can be published anywhere. (And I'm totally for open journals, even more, use CC-BY on it...)
vacri 15y ago
OSS aims for pragmatism Science aims for truth While there's a lot of overlap, there's a fundamental difference between the two, and methods to pursue pragmatism are not necessarily the right methods to pursue truth

by Dove 15y ago

adamjernst 15y ago

There's a thin line here. On one hand, you're right that openness is a good thing. On the other hand, modern science is incredibly political. When millions of dollars of funding is on the line, tensions run high. Anonymity is one way to shield reviewers. But the only way to have workable anonymous review is a trusted gatekeeper—and those are expensive.

lutorm 15y ago

I absolutely don't think that the current system means that the contents of journals are absolute truth -- I've been a referee and seen what happens to rejected articles (they show up a month later in another journal). The point is not so much about the quality of what does come out, it's about the quality of the crap that currently does not come out. While I agree that it could be useful to have open discussions about papers, I think if that happens at the expense of the ability of everyone who thinks they have an idea to put stuff out there, the result might be a net decrease in signal/noise.

jackpirate 15y ago

People propose that plan all the time. Here's why it can't work: While programmers like coding, researchers hate writing (they like doing research!). While there are relatively few OSS projects, there are millions of research papers (yes, millions! http://scholar.google.com/scholar?q=research&hl=en&btnG=Search&as_sdt=1%2C5 http://scholar.google.com/scholar?q=research&hl=en&b...) There are wiki style projects out there for each major domain. For example, quantum computing has http://www.quantiki.org http://www.quantiki.org. These sites are great, but they will never replace journal articles because they do not handle history or citations well. Imagine if all we had was wikipedia to tell us about the Church-Turing hypothesis? Papers like that give us an insight into the scientists' mind that nothing else can.

forkandwait 15y ago
It works now in physics: arxiv.org The reason it works is that your peers read your work, and either refute or cite you. Remember -- everybody knows everybody in a subfield, or at least everybody's advisor. It helps when there is math involved which is either correct or not. If somebody random submits something to arxiv, people know they don't know him or his grad school and that is thus probably a crackpot and so don't bother to read it (except sometimes someone from nowhere gets a result, which would NEVER be seen in the closed peer review process). EDIT: Also you attach your name to a finished product, not an unattributed ongoing wiki page. Your reputation is everything, so you win or lose based on the articles. One of the problems in social science academia is that 98% of the articles are completely worthless, and they only serve to get a professor promotion based on an index of citation times journal prestige. The smaller state colleges require their junior faculty to "publish", but they don't have enough to say to get into real journals and the publishing houses fill the need with "Journal of Rural Sociology" etc.
- jackpirate 15y ago
  IANA Physicist, but I can't imagine any physics journals feel threatened by arxiv. When someone wins a Nobel prize for work that was published exclusively through arxiv, then you might have a point.
  - imrehg 15y ago
    I Am A Physicist, and researchers do use Arxiv quite often: pre-publication papers, or something that is written up and can iterated (unlike journals, Arxiv can store multiple version of the same paper). It is not peer reviewed but good for having feedback on things you are doing. Of course, publications from the "big guns" go straight to the big journals, and their peer review is more like rubber-stamp if you have the name on your author list. So the only difference is for the bureaucrats, who want you to show them "proper" papers to be able to move forward, the people doing the actual research only care whether your experiment and explanation is solid or not, can be published anywhere. (And I'm totally for open journals, even more, use CC-BY on it...)

vacri 15y ago

OSS aims for pragmatism Science aims for truth While there's a lot of overlap, there's a fundamental difference between the two, and methods to pursue pragmatism are not necessarily the right methods to pursue truth