4 ms·

The journals published by the Royal Society, where modern enlightenment science was born, was informally peer reviewed by the mid 1700's, and peer review was sy

by kobeya 9y ago

The journals published by the Royal Society, where modern enlightenment science was born, was informally peer reviewed by the mid 1700's, and peer review was systematized in the 1800's. The explosive growth of the industrial revolution, the reordering of society in the enlightenment, and everything we call modern and progressive is associated with a sequence of events that is also tightly correlated with the birth of peer review. Most philosophers of science don't consider this a coincidence. Science gave birth to exponential progress, and science itself is dependent on peer review. Maybe the integration of peer review into practical industries, medical studies, and government regulation came later. But those were cases of taking something that worked astonishingly well and apply it to other domains. It's hardly fair to say that peer review "started" in the 1960's or 70's.

GalacticDomin8r 9y ago
The blog indicates this quite clearly and is speaking of only the formal, modern and monetized usage of it. Science itself is basically useless without some type of peer review in whatever form.
- kobeya 9y ago
  What I'm taking issue with is this: > But even if peer review is ancient, "peer review" itself is quite new. Google charts about the history of usage of a particular grouping of terms is not definitive evidence about the history of the underlying activity named by that phrase. Peer review is indeed ancient and fundamental to science and scientific progress. It just may not have been talked about as broadly or referred to under different names before then. It's hard to draw more than linguistic trivia out of that. It's especially bad to make policy suggestions based on that.
  - cwyers 9y ago
    But that's the author's point. This current thing we call peer review isn't what's fundamental to science, the broader concept is. The modern for-profit journal system is not the only way to do science, and just because it cloaks itself in the name "peer review" doesn't make it the only sort of peer review that facilitates good science.
    - trapperkeeper74 9y ago
      There are three activities: - critical feedback on papers - hypermonetization (and deleterious siloing) of works often produced with government grants, leading to primarily academics only having ready access to papers, whom don’t see the subscription costs as a burden to them. - disseminating only papers meeting a threshold to maintain quality of the journal Maybe scientists and academics should seek to create and use equivalent quality peer / committee adversalism but publish in open forum that operates based on donations. Academia seems currently over-commercialised: industry getting cheap labor, undergrads going into debt and journals, law libraries and textbook companies making a killing by locking up knowledge behind a paywall. Research papers and case law should be available to the most people as possible for little or no cost.
      - dnautics 9y ago
        Why the emphasis on money? The structural organization of science as a hierarchical enterpise with gatekeepers to advancement will result in these problems regardless of funding or commercialization.
        AstralStorm 9y ago
        Tying funding to it makes them much more acute though. Essentially forcing you to play by these rules in order to live. Without being forced like this, many deadbeat fake journals would die and places like arxiv would flourish.
        marcosdumay 9y ago
        That organization itself is a consequence of the funding policy. There's nothing else pushing scientists into a hierarchy, or at least nothing nearly as strong.
    - kobeya 9y ago
      Maybe I've missed the author's point, but you've also missed mine. The fact that the phrase "peer review" dates to the 1970's doesn't mean the concept wasn't in use before. It was just called something else ("refereeing" as someone else in this thread pointed out). Also, the sharp discontinuity in 1970-1980 is suspiciously close to the wide-scale deployment of computers. Probably has a lot more to do with data-availability biases in the corpus used by Google to construct these ngrams data sets from the transition to digital in industry and publishing.
  - KGIII 9y ago
    Latin and French were the languages of science for quite a while. Arabic before that and Greek before that. This is ignoring Eastern science intentionally. They searched for 'peer review.' I don't see any indication that they searched in other languages, what those phrases might be, and how they accounted for that. Well, that and it looks like they are conflating two ideas. They appear to be speaking about more modern journals and ignoring that philosophers shared and published their work, sought students, and held their findings up for scrutiny. So, not just the phrase part that you mention, but also the language used may impact this. I have no idea how to say peer review in either French or Latin and I am a scientist.
    - kobeya 9y ago
      Also German, for a short but meaningful period of time at least. Most of modern physics was originally published in German, for example.
pnathan 9y ago
Just for the sake of solidity, would you care to have a citation there?
- kobeya 9y ago
  Ironically there's some given in the article.
- synthmeat 9y ago
  Isaac Asimov’s New Guide to Science.
raverbashing 9y ago
The key word is: informally Formal peer review seems to encourage collective backscratching and discourage new ideas (even though some places do anonymous PR, it's not hard to guess where it's coming from if you're a specialist on the area and familiar with the most prominent labs)
- montecarl 9y ago
  I can confirm that it is very easy to determine who your anonymous reviewers are in specialist areas. In one topic that I used to publish in, I could easily identify one or two of the three reviewers. The main tells are when the reviewer brings up their tangentially related niche interests (e.g. how does this compare to method X or have you considered the effects of <relatively obscure phenomenon>).
  - kobeya 9y ago
    Also in some cases (e.g. many conferences) you can know who the list of reviewers are. Just look up the program committee. Only a subset reviews your paper submission, but with the field of possible reviewers constrained to 10-12 people you personally know, it becomes very hard to hide.

by kobeya 9y ago

GalacticDomin8r 9y ago

The blog indicates this quite clearly and is speaking of only the formal, modern and monetized usage of it. Science itself is basically useless without some type of peer review in whatever form.

kobeya 9y ago
What I'm taking issue with is this: > But even if peer review is ancient, "peer review" itself is quite new. Google charts about the history of usage of a particular grouping of terms is not definitive evidence about the history of the underlying activity named by that phrase. Peer review is indeed ancient and fundamental to science and scientific progress. It just may not have been talked about as broadly or referred to under different names before then. It's hard to draw more than linguistic trivia out of that. It's especially bad to make policy suggestions based on that.
- cwyers 9y ago
  But that's the author's point. This current thing we call peer review isn't what's fundamental to science, the broader concept is. The modern for-profit journal system is not the only way to do science, and just because it cloaks itself in the name "peer review" doesn't make it the only sort of peer review that facilitates good science.
  - trapperkeeper74 9y ago
    There are three activities: - critical feedback on papers - hypermonetization (and deleterious siloing) of works often produced with government grants, leading to primarily academics only having ready access to papers, whom don’t see the subscription costs as a burden to them. - disseminating only papers meeting a threshold to maintain quality of the journal Maybe scientists and academics should seek to create and use equivalent quality peer / committee adversalism but publish in open forum that operates based on donations. Academia seems currently over-commercialised: industry getting cheap labor, undergrads going into debt and journals, law libraries and textbook companies making a killing by locking up knowledge behind a paywall. Research papers and case law should be available to the most people as possible for little or no cost.
    - dnautics 9y ago
      Why the emphasis on money? The structural organization of science as a hierarchical enterpise with gatekeepers to advancement will result in these problems regardless of funding or commercialization.
      - AstralStorm 9y ago
        Tying funding to it makes them much more acute though. Essentially forcing you to play by these rules in order to live. Without being forced like this, many deadbeat fake journals would die and places like arxiv would flourish.
      - marcosdumay 9y ago
        That organization itself is a consequence of the funding policy. There's nothing else pushing scientists into a hierarchy, or at least nothing nearly as strong.
  - kobeya 9y ago
    Maybe I've missed the author's point, but you've also missed mine. The fact that the phrase "peer review" dates to the 1970's doesn't mean the concept wasn't in use before. It was just called something else ("refereeing" as someone else in this thread pointed out). Also, the sharp discontinuity in 1970-1980 is suspiciously close to the wide-scale deployment of computers. Probably has a lot more to do with data-availability biases in the corpus used by Google to construct these ngrams data sets from the transition to digital in industry and publishing.
- KGIII 9y ago
  Latin and French were the languages of science for quite a while. Arabic before that and Greek before that. This is ignoring Eastern science intentionally. They searched for 'peer review.' I don't see any indication that they searched in other languages, what those phrases might be, and how they accounted for that. Well, that and it looks like they are conflating two ideas. They appear to be speaking about more modern journals and ignoring that philosophers shared and published their work, sought students, and held their findings up for scrutiny. So, not just the phrase part that you mention, but also the language used may impact this. I have no idea how to say peer review in either French or Latin and I am a scientist.
  - kobeya 9y ago
    Also German, for a short but meaningful period of time at least. Most of modern physics was originally published in German, for example.

pnathan 9y ago

Just for the sake of solidity, would you care to have a citation there?

kobeya 9y ago
Ironically there's some given in the article.
synthmeat 9y ago
Isaac Asimov’s New Guide to Science.

raverbashing 9y ago

The key word is: informally Formal peer review seems to encourage collective backscratching and discourage new ideas (even though some places do anonymous PR, it's not hard to guess where it's coming from if you're a specialist on the area and familiar with the most prominent labs)

montecarl 9y ago
I can confirm that it is very easy to determine who your anonymous reviewers are in specialist areas. In one topic that I used to publish in, I could easily identify one or two of the three reviewers. The main tells are when the reviewer brings up their tangentially related niche interests (e.g. how does this compare to method X or have you considered the effects of <relatively obscure phenomenon>).
- kobeya 9y ago
  Also in some cases (e.g. many conferences) you can know who the list of reviewers are. Just look up the program committee. Only a subset reviews your paper submission, but with the field of possible reviewers constrained to 10-12 people you personally know, it becomes very hard to hide.