5 ms·

I think the peer review system is ripe for, as SV people call it, "disruption". Even something like an app that you submit papers to, which then anonymizes the

by typon 7y ago

I think the peer review system is ripe for, as SV people call it, "disruption". Even something like an app that you submit papers to, which then anonymizes them, standardizes formatting and presents them to randomly selected experts for review in a certain time period would go a long way compared to the current process. In my lab at a prestigious Canadian university, the papers' authors names were visible as well as the reviewers names because any back and forth post review would happen through email. I think journals and universities need to buy into a system that standardizes this process so at least these basic requirements of impartiality can be met.

tastroder 7y ago
While I'd apreciate a little fresh air in the process as well... > an app that you submit papers to, which then anonymizes them, standardizes formatting and presents them to randomly selected experts for review in a certain time period Except for that it's not an app (that would be kind of pointless, why would I want that on a phone instead of a webapp), that's pretty much the standard in my field. Anonymization is done through the authors (if the conference/journal wants it to be) and ensured by the conference submission system. Formatting is standardized anyway. The problem is the "randomly selected experts" part, for many (most?) subfields suitable reviewers are a scarce resource, to the point that in quite a few double-blind processes I've seen one or both parties could recognize each other's work or review. I've always found openreview [1] a nice way to at least make the process public, especially when anonymity is an illusion anyway. https://openreview.net/ https://openreview.net/
- typon 7y ago
  By app I meant software not a phone app. Secondly, I think the problem of having few experts in the field is a legitimate problem but I don't think the status quo is a good solution either. I think it erodes trust in the scientific community which is very hard to build back up.
  - tastroder 7y ago
    I think we need an overall shift in incentives in academia before that will change. Popular conferences see an all time high in submissions, though I do not have data on that I'm sure Journals face similar problems these days. I feel like that's not an aspect that will get better in the short term. On the app/phone app thing: Ah, fair enough, sorry about that.

by typon 7y ago

tastroder 7y ago

While I'd apreciate a little fresh air in the process as well... > an app that you submit papers to, which then anonymizes them, standardizes formatting and presents them to randomly selected experts for review in a certain time period Except for that it's not an app (that would be kind of pointless, why would I want that on a phone instead of a webapp), that's pretty much the standard in my field. Anonymization is done through the authors (if the conference/journal wants it to be) and ensured by the conference submission system. Formatting is standardized anyway. The problem is the "randomly selected experts" part, for many (most?) subfields suitable reviewers are a scarce resource, to the point that in quite a few double-blind processes I've seen one or both parties could recognize each other's work or review. I've always found openreview [1] a nice way to at least make the process public, especially when anonymity is an illusion anyway. https://openreview.net/ https://openreview.net/

typon 7y ago
By app I meant software not a phone app. Secondly, I think the problem of having few experts in the field is a legitimate problem but I don't think the status quo is a good solution either. I think it erodes trust in the scientific community which is very hard to build back up.
- tastroder 7y ago
  I think we need an overall shift in incentives in academia before that will change. Popular conferences see an all time high in submissions, though I do not have data on that I'm sure Journals face similar problems these days. I feel like that's not an aspect that will get better in the short term. On the app/phone app thing: Ah, fair enough, sorry about that.