9 ms·

"Indeed it has been said that Democ­racy is the worst form of Gov­ern­ment except for all those other forms that have been tried from time to time." -Winston C

• nmrm2 11y ago
  1 - Publicly available archival-quality publishing for all science done in the US would cost at least millions a year to administer. The idea is nice, but is just an idea without the funding to make it happen. 2 - Peer review problems could probably be addressed on a per-venue basis. Eg in many areas of CS the max turnaround time is a few months. And there's nothing stopping you from publishing peer reviewer's comments. But I am not convinced that enforcing this universally is a good idea, for either the author or the reviewer.
  • gnaritas 11y ago
    > Publicly available archival-quality publishing for all science done in the US would cost at least millions a year to administer. The idea is nice, but is just an idea without the funding to make it happen. This is what government is for and a million a year is a rounding error to the government.
    • nmrm2 11y ago
      Agreed. But I think this is important to point out. It's not as if scientists are lazy or greedy -- they spend the most time and get no benefit out of publications in journals in the status quo. There just aren't compelling alternatives to the traditional model in many disciplines, and most of the time it's due to want of desire rather than want of money.
  • nmrm2 11y ago
    Someone down voted this? Archival-grade hosting is pretty expensive -- it means the cost of publishing a paper is the cost of serving that paper (and keeping backups, maintaining the software infrastructure, etc) for... ever. Typically both for- and non-profits place this cost in the three-to-four figures range. It's not as simple as "slap it up on arxiv" -- arxiv would break in a second if every scientist started using it as the de facto platform for every paper.
• Retric 11y ago
  A reasonable compromise IMO is for Journals to have a very temporary lock on new publications. Public access after say 1 to 2 months seems to meet the vast majority of "Open Science" requirements without much change required. 2) IMO a much more stringent form of peer review is necessary. P<0.05 is only useful in very limited situations. I would argue that unless you republish your exact experimental design beforehand you would need to drop that to P<.01 to cover for a lot of marginal research out there. The old do 10 studies and you can likely publish one of them is harder when you need to do 50 or fudge the numbers harder.
  • munificent 11y ago
    > 2) IMO a much more stringent form of peer review is necessary. That disregards the cost of review. Reviewing and repeating experiments uses resources that could be spent elsewhere. On the long tail of papers that are not high impact even if correct, that's probably not a good allocation of effort. What I think would be better is that as a paper crawls up the impact curve (probably measured by citations), then additional resources should be spent validating and repeating it. In theory, that's how the system already works since everyone wants to build on top of or refute famous papers. In practice, I don't know if it does.
    • Retric 11y ago
      I am not suggesting people need to necessarily spend more time doing the review, just accept less BS. In many cases the bar is far to low.
    • bjelkeman-again 11y ago
      Repeating experiments is not a wasted effort. "More than half of biomedical findings cannot be reproduced – we urgently need a way to ensure that discoveries are properly checked" - Elizabeth Iorns is co-founder and CEO of Science Exchange. Science which can't be repeated, that is problematic. [1] http://www.newscientist.com/article/mg21528826.000-is-medical-science-built-on-shaky-foundations.html http://www.newscientist.com/article/mg21528826.000-is-medica...
  • saalweachter 11y ago
• DocSavage 11y ago
  It feels like we've lumped together a bunch of different use cases and metrics into the current publication system. 1) Authors need to get peer review, feedback to make their case stronger and improve the communication. 2) Researchers need a way of filtering the publications so they can find the gems in any field or ones very useful to their own work. 3) Then there's the need to evaluate how well particular people are doing in their research, which boils down to metrics including the prestige of the journal (immediate) and the number of citations it'll eventually accumulate (long term). I'm not aware of any distinction between citations of a work in an introduction or review and citations of a work that provided some essential part of the current method. These are pretty big differences since the former is more strongly influenced by reputation while the latter is its utility. Here's how one alternative system would work: Start with a github-like editing and annotation system for soliciting and coordinating feedback. This also allows readers and possible evaluation systems to see the types of contributions people make. References would get more sophisticated with categories like "background material", "thought provoking previous work", and "indispensable to this work." Publication initially goes to general repositories like arXiv where anyone can read it. Have an amped-up Google Scholar search where you can easily bookmark works of interest, tag them, and share them with colleagues. Different HN or reddit systems would spring up to bring better work to the attention of more researchers. "Journals" have a panel of editors who select a subset of work to display under their banner. This is not mutually exclusive so particularly awesome work would be featured in Nature, Science, and more specialized journals in this alternate universe. There would be a diversity of impact metrics including ones based purely on "my work would be impossible if not for this work X". Affiliation between authors could be factored into some metrics so greater impact is given to referenced work with more degrees of separation from the author's current and previous institutions.
  • semi-extrinsic 11y ago
    I like the concept of semantic citations, e.g. "cited for methodX", but it's hard to make a general enough system out of this that doesn't disrupt the flow while reading. And in a way, we already do it, except that it's not being done in a consistent fashion that enables datamining of it. Getting publishers to standardize some markup for this would be cool, but I don't think the outcome justifies the added effort for everyone. As for several journals publishing the same thing: there are many problems withh that idea, not the least that Nature/Science doesn't want your paper with an 11 page methods section, they want something much closer to a popsci article. So you have to write several. This is already happening today, when someone gets to publish their supercool finding in Nature they also write one or more longer, detailed (i.e. proper) papers that are sent to more specific journals.