4 ms·

I would say academics are just people, and in many cases they want to be employed and successful. At the moment, while the system is very bad in many ways, it

I would say academics are just people, and in many cases they want to be employed and successful. At the moment, while the system is very bad in many ways, it used to work (in my opinion, I haven't done proper research into this) because: * People who run "good" journals want their journal to be considered good, so they publish good papers. * People want to get into "good" journals, so they want to write good papers that will be accepted. The existence of these papers is then used as evidence for promotions and grants. * Academics want to read "good" journals, so they get their University to pay for a subscription to get physical copies in the library. Previously this cycle held reasonably well -- it's been broken by the reduced cost of online journals, and journals wanting higher payments. I think a good future system needs to (somehow) recreate this "virtuous cycle" -- making reviews as important as journals could be a good idea, I could imagine following a reviewer I thought was excellent, and a group of reviewers could create their own "virtual journal". I would personally trust "Person X who I know thinks this paper is good" much more than "100 people thought this paper was good, 40 thought it was bad, so +60".

dbingham 4y ago
Virtual journals is definitely on the table. The way I'm thinking it would work is that journals can effectively create a team of reviewers on the platform and when you submit a paper, you could request review from their team. Right now I'm thinking you could choose to request open review from everyone in your fields, review from one or more journals, or both at the same time. The journal editors then give their feedback through the review system, but it's still up to the author to choose to publish. If you successfully satisfy the journal, you get their stamp of approval on your paper when you publish. If not, you can still publish at will, but your risk their review team down voting you and you don't get the stamp of approval. > I would personally trust "Person X who I know thinks this paper is good" much more than "100 people thought this paper was good, 40 thought it was bad, so +60". I can definitely understand where that hesitation would come from - especially with experience in places like Reddit, where the crowd is not given guidelines and voting systems decay to the lowest common denominator. My understanding of the research on the topic is that it actually supports the idea that the judgement of the crowd will often be better than the judgement of any individual -- when it's the right crowd, given the right guidelines. That's the model behind StackExchange and it's worked very well there. And that's the theory behind this (proposed) platform. The reputation and field model is geared towards identifying and creating groups of "the right people" for each topic/discipline/field and allowing people to submit their work to those groups. It could create a similar virtual cycle: * People want to support good science and get good review feedback, so they give good review feedback. * People want to support and highlight good science, so they upvote good science, downvote dishonest science, and post responses with critical feedback on mediocre science. * People want to have their work to be upvoted and receive positive responses, so they take review feedback seriously. That's the theory anyway - the jury seems to be very much out on whether or not that can work with academia.

dbingham 4y ago

Virtual journals is definitely on the table. The way I'm thinking it would work is that journals can effectively create a team of reviewers on the platform and when you submit a paper, you could request review from their team. Right now I'm thinking you could choose to request open review from everyone in your fields, review from one or more journals, or both at the same time. The journal editors then give their feedback through the review system, but it's still up to the author to choose to publish. If you successfully satisfy the journal, you get their stamp of approval on your paper when you publish. If not, you can still publish at will, but your risk their review team down voting you and you don't get the stamp of approval. > I would personally trust "Person X who I know thinks this paper is good" much more than "100 people thought this paper was good, 40 thought it was bad, so +60". I can definitely understand where that hesitation would come from - especially with experience in places like Reddit, where the crowd is not given guidelines and voting systems decay to the lowest common denominator. My understanding of the research on the topic is that it actually supports the idea that the judgement of the crowd will often be better than the judgement of any individual -- when it's the right crowd, given the right guidelines. That's the model behind StackExchange and it's worked very well there. And that's the theory behind this (proposed) platform. The reputation and field model is geared towards identifying and creating groups of "the right people" for each topic/discipline/field and allowing people to submit their work to those groups. It could create a similar virtual cycle: * People want to support good science and get good review feedback, so they give good review feedback. * People want to support and highlight good science, so they upvote good science, downvote dishonest science, and post responses with critical feedback on mediocre science. * People want to have their work to be upvoted and receive positive responses, so they take review feedback seriously. That's the theory anyway - the jury seems to be very much out on whether or not that can work with academia.