3 ms·

> We should be trying to maximize the growth of knowledge, not minimizing the number of wrong papers, which is what max(s/n) leads to. IMO, filtering out noise

by papeda 6y ago

> We should be trying to maximize the growth of knowledge, not minimizing the number of wrong papers, which is what max(s/n) leads to. IMO, filtering out noise is a big part of maximizing the growth of knowledge. I cannot read every paper in my area that goes up on Arxiv, let alone every paper in related areas. That's why it's useful, albeit imperfect, to have gatekeeping peer-review organizations to force experts to look at papers and pick out the ones they think are most useful. There is a lot of noise on Arxiv.

barry-cotter 6y ago
> That's why it's useful, albeit imperfect, to have gatekeeping peer-review organizations to force experts to look at papers and pick out the ones they think are most useful Gate keeping and review do not require anything like the current peer review system. The current peer review system retards the flow of knowledge, or it would if anyone relied on journals to know the state of the field. Gate keeping can be done by editorial review. If you really want to you can have accept/reject only with no revise and resubmit cycle like Sociological Science. They aim for 30 day from submission to publication. Refereeing wastes more than $50m a year in value in Economics alone. Burn the system to the ground. > IZA DP No. 12866: Is Scholarly Refereeing Productive (at the Margin)? > In economics many articles are subjected to multiple rounds of refereeing at the same journal, which generates time costs of referees alone of at least $50 million. This process leads to remarkably longer publication lags than in other social sciences. We examine whether repeated refereeing produces any benefits, using an experiment at one journal that allows authors to submit under an accept/reject (fast-track or not) or the usual regime. We evaluate the scholarly impacts of articles by their subsequent citation histories, holding constant their sub-fields, authors' demographics and prior citations, and other characteristics. There is no payoff to refereeing beyond the first round and no difference between accept/reject articles and others. This result holds accounting for authors' selectivity into the two regimes, which we model formally to generate an empirical selection equation. This latter is used to provide instrumental estimates of the effect of each regime on scholarly impact. https://www.iza.org/publications/dp/12866/is-scholarly-refereeing-productive-at-the-margin https://www.iza.org/publications/dp/12866/is-scholarly-refer...
- papeda 6y ago
  I agree that the multi-year publication timelines for journals are in general silly. I agree that keeping up with a field by reading journals doesn't work. If by editorial review you mean "the editor alone decides", I don't think that's a good replacement. I think the computer science publication model of a few conferences per year with ~3 month submit-to-decision timelines, based on 3-5 reviews per paper, is pretty good. But it's probably not appropriate for fields with deep contributions that take substantial time to verify, like math.

by papeda 6y ago

barry-cotter 6y ago

> That's why it's useful, albeit imperfect, to have gatekeeping peer-review organizations to force experts to look at papers and pick out the ones they think are most useful Gate keeping and review do not require anything like the current peer review system. The current peer review system retards the flow of knowledge, or it would if anyone relied on journals to know the state of the field. Gate keeping can be done by editorial review. If you really want to you can have accept/reject only with no revise and resubmit cycle like Sociological Science. They aim for 30 day from submission to publication. Refereeing wastes more than $50m a year in value in Economics alone. Burn the system to the ground. > IZA DP No. 12866: Is Scholarly Refereeing Productive (at the Margin)? > In economics many articles are subjected to multiple rounds of refereeing at the same journal, which generates time costs of referees alone of at least $50 million. This process leads to remarkably longer publication lags than in other social sciences. We examine whether repeated refereeing produces any benefits, using an experiment at one journal that allows authors to submit under an accept/reject (fast-track or not) or the usual regime. We evaluate the scholarly impacts of articles by their subsequent citation histories, holding constant their sub-fields, authors' demographics and prior citations, and other characteristics. There is no payoff to refereeing beyond the first round and no difference between accept/reject articles and others. This result holds accounting for authors' selectivity into the two regimes, which we model formally to generate an empirical selection equation. This latter is used to provide instrumental estimates of the effect of each regime on scholarly impact. https://www.iza.org/publications/dp/12866/is-scholarly-refereeing-productive-at-the-margin https://www.iza.org/publications/dp/12866/is-scholarly-refer...

papeda 6y ago
I agree that the multi-year publication timelines for journals are in general silly. I agree that keeping up with a field by reading journals doesn't work. If by editorial review you mean "the editor alone decides", I don't think that's a good replacement. I think the computer science publication model of a few conferences per year with ~3 month submit-to-decision timelines, based on 3-5 reviews per paper, is pretty good. But it's probably not appropriate for fields with deep contributions that take substantial time to verify, like math.