3 ms·

The credibility problem has been partially solved with a PageRank type algorithm. (Sergei and Larry were actually inspired by the weighting system of research p

The credibility problem has been partially solved with a PageRank type algorithm. (Sergei and Larry were actually inspired by the weighting system of research papers) Highly regarded scientific journals like the Lancet are used to accelerate this weighting process. The problem is that nowadays they seem kind of outdated, opaque and corrupt on many levels. Nobody wants to suppress peer-review, simply replace the journals mafia with something less corrupt.

knzhou 6y ago
But this isn't a case of a journal acting like a "mafia". This is a case where something bad slipped past peer review, because of the inherent limitations of peer review. The reason is that expert time and attention is intrinsically scarce (if you double the number of experts, you double the number of papers produced, and hence double the amount of time that must be spent on peer review), and really properly vetting a paper takes a massive amount of time. This wouldn't be changed by system reform. In my opinion, the best option for everybody who's serious about this is to read a lot of papers, until you become "well-calibrated", i.e. when you have a good handle on the overall chances a preprint is true, vs. a Twitter thread, vs. a paper in a top-tier journal. This has to be learned from experience. Reform doesn't change the fact that you need to do this -- it just adds yet another venue, either more or less reliable than the previous ones, that you also need to become calibrated on.
- fragmede 6y ago
  > read a lot of papers In order to do this, those that are serious need to be able to access papers, but paying hundreds or thousands of dollars for Elsevier can put them out of reach.
  - knzhou 6y ago
    Indeed, but that is the problem that preprints can solve. (As in, they don't solve the problem of telling what is correct, because that's just inherently hard, but they solve the problem of giving access so people can learn to tell.)
- stephc_int13 6y ago
  There is also the problem with cost and access. (Remember Aaron Swartz) https://en.wikipedia.org/wiki/Open_access https://en.wikipedia.org/wiki/Open_access https://en.wikipedia.org/wiki/Aaron_Swartz https://en.wikipedia.org/wiki/Aaron_Swartz

knzhou 6y ago

But this isn't a case of a journal acting like a "mafia". This is a case where something bad slipped past peer review, because of the inherent limitations of peer review. The reason is that expert time and attention is intrinsically scarce (if you double the number of experts, you double the number of papers produced, and hence double the amount of time that must be spent on peer review), and really properly vetting a paper takes a massive amount of time. This wouldn't be changed by system reform. In my opinion, the best option for everybody who's serious about this is to read a lot of papers, until you become "well-calibrated", i.e. when you have a good handle on the overall chances a preprint is true, vs. a Twitter thread, vs. a paper in a top-tier journal. This has to be learned from experience. Reform doesn't change the fact that you need to do this -- it just adds yet another venue, either more or less reliable than the previous ones, that you also need to become calibrated on.

fragmede 6y ago
> read a lot of papers In order to do this, those that are serious need to be able to access papers, but paying hundreds or thousands of dollars for Elsevier can put them out of reach.
- knzhou 6y ago
  Indeed, but that is the problem that preprints can solve. (As in, they don't solve the problem of telling what is correct, because that's just inherently hard, but they solve the problem of giving access so people can learn to tell.)
stephc_int13 6y ago
There is also the problem with cost and access. (Remember Aaron Swartz) https://en.wikipedia.org/wiki/Open_access https://en.wikipedia.org/wiki/Open_access https://en.wikipedia.org/wiki/Aaron_Swartz https://en.wikipedia.org/wiki/Aaron_Swartz