14 ms·

A lot of the replies make it seem like there is some great over-arching coordination and intent between subsequent submissions, but I’ll offer up an alternative

• grepLeigh 2y ago
  What's the compensation scheme for reviewers? Are there any mechanisms to balance out the "race to the bottom" observed in other types of academic compensation? e.g. increase of adjunct/gig work replacing full-time professorship. Do universities require staff to perform a certain number of reviews in academic journals?
  • acomjean 2y ago
    I know from some of my peers that reviewed biology (genetics) papers, they weren’t compensated. I was approached to review something for no compensation as well, but I was a bad fit.
  • hanche 2y ago
    Normally, referees are unpaid. You're just supposed to do your share of referee work. And then the publisher sells the fruits of all that work (research and refereeing) back to universities at a steep price. Academic publishing is one of the most profitable businesses on the planet! But univesities and academics are fighting back. Have been for a few years, but the fight is not yet over.
    • throwaway2037 2y ago
      If unis "win", what is the likely outcome?
      • bumby 2y ago
        More/easier/cheaper dissemination of research.
  • paulpauper 2y ago
    It's implicitly understood that volunteer work makes the publishing process 'work'. It's supposed to be a level playing field where money does not matter.
  • davrosthedalek 2y ago
    Typically, at least in physics (but as far as I know in all sciences), it's not compensated, and the reviewers are anonymous. Some journals try to change this, with some "reviewer coins", or Nature, which now publishes reviewer names if a paper is accepted and if the reviewer agrees. I think these are bad ideas. Professors are expected to review by their employer, typically, and it's a (very small) part of the tenure process.
  • tokinonagare 2y ago
    I don't thing it's a money problem. It's more like a framing issue, with some reviewers being too narrow-minded, or lacking background knowledge on the topic of the paper. It's not uncommon to have a full lab with people focussing on very different things, when you look in the details, the exact researchers interests don't overlap too much.
  • canjobear 2y ago
    There is no compensation for reviewers, and usually no compensation for editors. It’s effectively volunteer work. I agree to review a paper if it seems interesting to me and I want to effectively force myself to read it a lot more carefully than normal. It’s hard work, especially if there is a problem with the paper, because you have to dig out the problem and explain it clearly. An academic could refuse to do any reviews with essentially no formal consequences, although they’d get a reputation as a “bad citizen” of some kind.
  • SJC_Hacker 2y ago
    > Do universities require staff to perform a certain number of reviews in academic journals? No. Reviewers mostly do it because its expected of them, and they want to publish their own papers so they can get grants In the end, the university only cares about the grant (money), because they get a cut - somewhere between 30-70% depending on the instituition/field - for "overhead" Its like the mafia - everyone has a boss they kick up to. My old boss (PI on an RO1) explained it like this Ideas -> Grant -> Money -> Equipment/Personnel -> Experiments -> Data -> Paper -> Submit/Review/Publish (hopefully) -> Ideas -> Grant If you don't review, go to conferences/etc. its much less likely your own papers will get published, and you won't get approved for grants. Sadly there is still a bit of "junior high popularity contest" , scratch my back I'll scratch yours that is still present in even "highly respected" science journals. I hear this from basically every scientist I've known. Even successful ones - not just the marginal ones.
    • davrosthedalek 2y ago
      While most of what you write is true to some extend, I do not see how reviewing will get your paper published, except maybe for the cases the authors can guess the reviewer. It's anonymous normally.
      • SJC_Hacker 2y ago
        The editor does though, they all know each other. They would know who's not refereeing - and word gets around.
      • deleted 2y ago
        [deleted]
  • jasonfarnon 2y ago
    Do universities require staff to perform a certain number of reviews in academic journals? Depends on what you mean by "require". At most research universities it is a plus when reviewing tenureship files, bonuses, etc. It is a sign that someone cares about your work, and the quality of the journal seeking your review matters. If it were otherwise faculty wouldn't list the journals they have reviewed for on their CVs. If no one would ever find out about a reviewers' efforts e.g. the process were double blind to everyone involved, the setup wouldnt work.
• hanche 2y ago
  > sometimes the reviewer selection is an utter crap shoot Indeed, but when someone of Tao's caliber submits a paper, any editor would (should) make an extra effort to get the very best researchers to referee the paper.
  • httpsterio 2y ago
    depending on the publication the reviewers might not even know who the authors are.
    • sharth 2y ago
      But the journal editor should.
  • crote 2y ago
    But isn't that exactly why the submission should be anonymous to the reviewer? It's science, the paper should speak for itself. You don't want a reviewer to be biased by the previous accomplishments of the author. An absolute nobody can make groundbreaking and unexpected discoveries, and a Nobel prize winner can make stupid mistakes.
    • hoten 2y ago
      The reviewer wouldn't need to know, just the one coordinating who should review what.
      • sokoloff 2y ago
        Inherent in the editor trying to "get the very best researchers to [review] the paper" is likely to be a leak of signal. (My spouse was a scientific journal editor for years; reviewers decline to review for any number of reasons, often just being too busy and the same reviewer is often asked multiple times per year. Taking the extra effort to say "but this specific paper is from a really respected author" would be bad, but so would "but please make time to review this specific paper for reasons that I can't tell you".)
• foxglacier 2y ago
  Or maybe it doesn't matter. He got them published anyway and just lost some prestigious journal points on his career. Science/math was the winner on the day and that's the whole point of it. Maybe some of those lower ranked journals are run better and legitimately chipping away at the prestige of the top ones due to their carelessness.
  • bumby 2y ago
    Research and publication incur opportunity costs. For every manuscript that has to be reworked and submitted elsewhere, you’re losing the ability to do new research. So a researcher is left trying to balance the cost/benefit of additional time investment. Sometimes that results in a higher quality publication, sometimes it results in abandoning good (or bad) work, and sometimes it just wastes time.
    • melagonster 2y ago
      foxglacier offered a very good point! If some guy is so talented as Tao, perhaps this is the time to ameliorate journal by his power (like what he did here).
• wrsh07 2y ago
  Right - it's somewhat similar to code review Sometimes one person is looking for an improvement in this area while someone else cares more about that other area This is totally reasonable! (Ideally if they're contradicting each other you can escalate to create a policy that prevents future contradictions of that sort)
• dhosek 2y ago
  Luck plays a lot of a role in many vaguely similar things. I regularly submit fiction and poetry for publication (with acceptance rates of 2% for fiction and 1.5% for poetry) and so much depends on things well out of my control (which is part of why I’m sanguine about those acceptance rates—given the venues I‘m submitting to, they’re not unreasonable numbers and more recent years’ stats are better than that).¹ In many cases the editors like what they read, but don’t have a place for it in the current issue or sometimes they’re just having a bad day. ⸻ 1. For those who care about the full messy details I have charts and graphs at https://www.dahosek.com/2024-in-reejctions-and-acceptances/ https://www.dahosek.com/2024-in-reejctions-and-acceptances/
• keepamovin 2y ago
  And this is how we do science? How is that a good basis for scientific reality? Seems there should at least be transparency and oversight, or maybe the whole system is broke: open reviews on web not limited to a small committee sounds better. Science is about the unknown, building testable models and getting data. Even an AI review system could help.
  • n144q 2y ago
    It is not a good way of doing science, but it is the best we have. All the alternatives, including the ones you proposed, have their own serious downsides, which is why we kept the status quo for the past few decades.
    • fastball 2y ago
      What is the serious downside of open internet centric review?
      • reilly3000 2y ago
        The open internet. i.e. trolls, brigades, spammers, bots, and all manner of uninformed voices.
        • bruce511 2y ago
          To expand on this - because if the barrier to publishing is zero, then the "reputation" of the publisher is also zero. (Actually, we already have the "open publishing" you are suggesting - it's called Blogging or social media.) In other words, if we have open publishing, then someone like me (with zero understanding of a topic) can publish a very authentic-looking pile of nonsense with exactly the same weight as someone who, you know, has actually done some science and knows what they're talking about. The common "solution" to this is voting - like with StackOverflow answers. But that is clearly trivial to game and would quickly become meaningless. So human review it is - combined with the reputation that a journal brings. The author gains reputation because some reviewers (with reputation) reviewed the paper, and the journal (with reputation) accepted it. Yes, this system is cumbersome, prone to failure, and subject to outside influences. It's not perfect. Just the best we have right now.
• Salgat 2y ago
  This is all due to the preverse incentives of modern academia prioritizing quantity over quantity, flooding journals with an unending churn of low effort garbage.
  • bruce511 2y ago
    There are easily tens of thousands of researchers globally. If every one did a single paper per year, that would still be way more than journals could realistically publish. Since it is to some extent a numbers game, yes, academics (especially newer ones looking to build reputation) will submit quantity over quality. More tickets in the lottery means more chances to win. I'm not sure though how you change this. With so many voices shouting for attention it's hard to distinguish "quality" from the noise. And what does it even mean to prioritize "quality"? Is science limited to 10 advancements per year? 100? 1000? Should useful work in niche fields be ignored simply because the fields are niche? Is it helpful to have academics on staff for multiple years (decades?) before they reach the standard of publishing quality? I think perhaps the root of the problem you are describing is less one of "quantity over quality" and more one of an ever-growing "industry" where participants are competing against more and more people.
    • eru 2y ago
      > [...] way more than journals could realistically publish. In what sense? If you put it on a website, you can publish a lot more without breaking a sweat. People who want a dead tree version can print it out on demand.
      • bruce511 2y ago
        Publishing in the sense or reviewing, editing, etc. Distribution is the easy part.
        • eru 2y ago
          Well, but that scales with the number of people. The scientists themselves are working as reviewers. More scientists writing papers also means more scientists available for reviewing papers. And as you say, distribution is easy, so you can do reviewing after publishing instead of doing it before.
• nine_k 2y ago
  It's as if big journals are after some drama. Or excitement at least. Not just an important result, but a groundbreaking result in its own right. If it's a relatively small achievement that finishes a long chain of gradual progress, it better be some really famous problem, like Fermat's last theorem, Poinrcaré's conjecture, etc. I wonder if it's actually optimal from the journal's selfish POV. I would expect it to want to publish articles that would be cited most widely. These should be results that are important, that is, are hubs for more potential related work, rather that impressive but self-contained results.