4 ms·
I really think there needs to be some deep reexamination of the current way we quanitfy the value of research output in the academic world. I don't have the tim
by dwrodri 3y ago
I really think there needs to be some deep reexamination of the current way we quanitfy the value of research output in the academic world. I don't have the time or energy to develop a full argument in favor of an alternative replacement, but I'll do my best to share a "cut down" version.
Some of the biggest problems with the current system are:
* Peer Review has its share of problems[1][2][3] that create horrible second order effects, especially among those pursuing PhDs[4]
* Elsevier, Nature, Science/AAAS and others perform rent-seeking behavior to the extent where I think it's worth asking whether they hinder the funding and dissemination of good science more than they help. As a personal aside, I always found it very off-putting that DeepMind regularly publishes in Nature in Science, despite the fact that outside of AlphaFold, their work often has little overlap with the readership that typically frequents these journals.
Personally, I am of the opinion that platforms like Semantic Scholar, arXiv and OpenReview are doing a better job of promoting open and transparent academic research with improved accessibility to both the public and the researchers doing good work.
Given the power of being mentored by great scientists, it makes sense to have filtering processes which concentrate great researchers in a small amount of schools. My point is that if there is too little oversight, these institutions become incentivized to all but encourage bad behavior in order to maintain their image. We need systems which encourage MORE transparency into the process of creating science and MORE accessibility because an important part of scientific research is it's uncertainty.
Tools like arXiv, Semantic Scholar, and OpenReview are all steps in the right direction, and it would be good to promote the useage of these tools outside of their current userbase as I think they provide a system for people to observe science more easily, and for important parts of the research process to be accessed by all.
1: https://academia.stackexchange.com/questions/115231/why-is-peer-review-so-random https://academia.stackexchange.com/questions/115231/why-is-p...
2: https://blog.neurips.cc/2021/12/08/the-neurips-2021-consistency-experiment/ https://blog.neurips.cc/2021/12/08/the-neurips-2021-consiste...
3: https://jamanetwork.com/journals/jama/article-abstract/194989 https://jamanetwork.com/journals/jama/article-abstract/19498...
4: https://medium.com/@tnvijayk/potential-organized-fraud-in-acm-ieee-computer-architecture-conferences-ccd61169370d https://medium.com/@tnvijayk/potential-organized-fraud-in-ac...