3 ms·

> Tenure precedes peer review afaik Um, no. Tenure decisions turn largely on publication record, which turns on peer review. > Citations exist independently

by lisper 2y ago

> Tenure precedes peer review afaik Um, no. Tenure decisions turn largely on publication record, which turns on peer review. > Citations exist independently of peer review To cite something there has to be something to cite. It is of course possible to cite a non-peer-reviewed publication, but in academia this is heavily frowned upon. Non-peer-reviewed is generally considered synonymous with crackpottery. > Pick 10 random bureaucrats I meant do you think it's possible to "shmooze [your] way onto the grant review board" without a publication record in the real world, not some counterfactual world where you have stacked the deck. > the replication crises has many origins and isn’t just the fault of peer reviews I didn't say it was just the fault of peer review. What I said was that peer review was "probably in no small measure responsible for [the] replication crises", and I stand by that.

JadeNB 2y ago
> > Tenure precedes peer review afaik > Um, no. Tenure decisions turn largely on publication record, which turns on peer review. I'm pretty sure your parent meant that the concept of tenure precedes the concept of peer review. However, this too seems to be false, according to the repository of truth, Wikipedia, which says that: > The first record of an editorial pre-publication peer-review is from 1665 by Henry Oldenburg, the founding editor of Philosophical Transactions of the Royal Society at the Royal Society of London. (https://en.wikipedia.org/wiki/Scholarly_peer_review https://en.wikipedia.org/wiki/Scholarly_peer_review) but: > Tenure was introduced into American universities in the early 1900s in part to prevent the arbitrary dismissal of faculty members who expressed unpopular views. (https://en.wikipedia.org/wiki/Academic_tenure https://en.wikipedia.org/wiki/Academic_tenure).
- lisper 2y ago
  > I'm pretty sure your parent meant that the concept of tenure precedes the concept of peer review. Even if that were true, what does the historical development of these institutions have to do with the claim that contemporary peer review is responsible for the contemporary replication crisis?
  - vlovich123 2y ago
    Yeah I obviously am taking about the peer reviewed journal not peer review as a concept (which is how this discussion started). ~~But it does look like tenure is after journals not before.~~ Correction: peer review as we know of began in the mid 1970s, so tenure precedes the modern peer review system.
vlovich123 2y ago
Ok, so what you seem to have said in this thread is that the system today is a huge contributor to the replication crisis but any suggestion that things could be done differently is met with an endless barrage of questions and resistance that no, it had to be done this way. So your complaint is that there’s a system at all instead of anarchy? Really not sure what point you’re trying to make.
- lisper 2y ago
  Maybe re-reading my original comment will help clarify: https://news.ycombinator.com/item?id=40456188 https://news.ycombinator.com/item?id=40456188
  - vlovich123 2y ago
    Not really no. I suggested that you can decouple tenure (1900s) from modern peer review (1970s). Citations aren't an issue when publishing (you can publish anywhere) a result but are maybe more of an issue when you have a collaborative body of work (e.g. closer to open-source software development). But even still you can have citations (e.g. the citation using the Discord channel). For some reason you seemed to take the position that citations are inextractable from peer review. The grant mechanism is definitely a problem because of how it interacts with university funding, but the grant selection mechanism can evolve to be closer to how entrepreneurs work in the business market (which has its own pluses and minuses). What I suggested though is that even if you completely remove the modern peer review system, you'll still be left with the replication crises because. You've seem to have taken issue both with the suggestion that peer review is removable from academia and completely failed to engage with the issues that have nothing to do with peer review. 1. Issues around funding are a higher order problem with peer review only being a symptom at best (if at all). For example, Sabine talks about the issues and focuses on grants and spends 0 time on modern peer review. 2. Fraud didn't come into being because of peer review but grows with funding. The more funding the bigger the problem. Conversely the less funding the more likely proportionally the research is fraudulent or of poor quality because there's a smaller community checking the work. We know that the more funding we spend, the more research activity a field experiences. We don't have good ways to sift out fraud proactively - it takes disproportionately more work to root out fraud and bad science than it is to publish that & reap the rewards. This is true beyond academia - it's easier to spew BS than it is to explain the truth. 3. Not registering for null results has nothing to do with peer review. It's more just "hey I did this work and I'm not going to get rewarded so I'm not going to bother spending the work to publish the null result". That exists independent of the modern peer review system & even publish/perish is ancillary to this - a null result amounts to "failure" emotionally and that can be hard to deal with. That's why there's systems now to mandate pre-registration of the experiment - so that meta analysis can determine whether or not a result has actually been replicated enough to reduce the risk of p-hacking. 4. The replication crises for particle physics is a stark example how peer review is not really contributing as much. There's two schools of thought. The first is that we just follow the math and use data to refine which mathematical model to pick. The second is that we need to come up with better philosophical underpinnings for what the math is telling us. For now the first school is winning in terms of funding dollars (& results), but it's really hard to determine a priori which path is actually the one we should be following. Moreover, the orthodoxy exists independent of the peer review system (& even independent of grant proposals).

by lisper 2y ago

JadeNB 2y ago

> > Tenure precedes peer review afaik > Um, no. Tenure decisions turn largely on publication record, which turns on peer review. I'm pretty sure your parent meant that the concept of tenure precedes the concept of peer review. However, this too seems to be false, according to the repository of truth, Wikipedia, which says that: > The first record of an editorial pre-publication peer-review is from 1665 by Henry Oldenburg, the founding editor of Philosophical Transactions of the Royal Society at the Royal Society of London. (https://en.wikipedia.org/wiki/Scholarly_peer_review https://en.wikipedia.org/wiki/Scholarly_peer_review) but: > Tenure was introduced into American universities in the early 1900s in part to prevent the arbitrary dismissal of faculty members who expressed unpopular views. (https://en.wikipedia.org/wiki/Academic_tenure https://en.wikipedia.org/wiki/Academic_tenure).

lisper 2y ago
> I'm pretty sure your parent meant that the concept of tenure precedes the concept of peer review. Even if that were true, what does the historical development of these institutions have to do with the claim that contemporary peer review is responsible for the contemporary replication crisis?
- vlovich123 2y ago
  Yeah I obviously am taking about the peer reviewed journal not peer review as a concept (which is how this discussion started). ~~But it does look like tenure is after journals not before.~~ Correction: peer review as we know of began in the mid 1970s, so tenure precedes the modern peer review system.

vlovich123 2y ago

Ok, so what you seem to have said in this thread is that the system today is a huge contributor to the replication crisis but any suggestion that things could be done differently is met with an endless barrage of questions and resistance that no, it had to be done this way. So your complaint is that there’s a system at all instead of anarchy? Really not sure what point you’re trying to make.

lisper 2y ago
Maybe re-reading my original comment will help clarify: https://news.ycombinator.com/item?id=40456188 https://news.ycombinator.com/item?id=40456188
- vlovich123 2y ago
  Not really no. I suggested that you can decouple tenure (1900s) from modern peer review (1970s). Citations aren't an issue when publishing (you can publish anywhere) a result but are maybe more of an issue when you have a collaborative body of work (e.g. closer to open-source software development). But even still you can have citations (e.g. the citation using the Discord channel). For some reason you seemed to take the position that citations are inextractable from peer review. The grant mechanism is definitely a problem because of how it interacts with university funding, but the grant selection mechanism can evolve to be closer to how entrepreneurs work in the business market (which has its own pluses and minuses). What I suggested though is that even if you completely remove the modern peer review system, you'll still be left with the replication crises because. You've seem to have taken issue both with the suggestion that peer review is removable from academia and completely failed to engage with the issues that have nothing to do with peer review. 1. Issues around funding are a higher order problem with peer review only being a symptom at best (if at all). For example, Sabine talks about the issues and focuses on grants and spends 0 time on modern peer review. 2. Fraud didn't come into being because of peer review but grows with funding. The more funding the bigger the problem. Conversely the less funding the more likely proportionally the research is fraudulent or of poor quality because there's a smaller community checking the work. We know that the more funding we spend, the more research activity a field experiences. We don't have good ways to sift out fraud proactively - it takes disproportionately more work to root out fraud and bad science than it is to publish that & reap the rewards. This is true beyond academia - it's easier to spew BS than it is to explain the truth. 3. Not registering for null results has nothing to do with peer review. It's more just "hey I did this work and I'm not going to get rewarded so I'm not going to bother spending the work to publish the null result". That exists independent of the modern peer review system & even publish/perish is ancillary to this - a null result amounts to "failure" emotionally and that can be hard to deal with. That's why there's systems now to mandate pre-registration of the experiment - so that meta analysis can determine whether or not a result has actually been replicated enough to reduce the risk of p-hacking. 4. The replication crises for particle physics is a stark example how peer review is not really contributing as much. There's two schools of thought. The first is that we just follow the math and use data to refine which mathematical model to pick. The second is that we need to come up with better philosophical underpinnings for what the math is telling us. For now the first school is winning in terms of funding dollars (& results), but it's really hard to determine a priori which path is actually the one we should be following. Moreover, the orthodoxy exists independent of the peer review system (& even independent of grant proposals).