5 ms·

I agree completely. In my experience as a PhD student (comp sci), all conferences make reviewers anonymous. As a result, there is very little accountability reg

by seccess 12y ago

I agree completely. In my experience as a PhD student (comp sci), all conferences make reviewers anonymous. As a result, there is very little accountability regarding the reviews you receive for your work. More than once, I have had papers rejected simply because a single reviewer barely read the paper and dismissed it. These kinds of reviews are very frustrating to receive, not only because they failed to understand the basic premises of your paper, but because these reviews contain no useful information on how to make your paper better for future submissions. Of course, the opposite can happen where mediocre works slips through, and the reviewers that allowed that should be held accountable too. Its painful to me that so much of the acceptance process for research papers (in my field at least) is based on luck. Moral of the story: reviewer anonymity is good but it comes at the expensive of accountability.

javert 12y ago
I'm also a computer science PhD student. I don't disagree with anything you said, but I am even more cynical. > Of course, the opposite can happen where mediocre works slips through Slips through? Mediocre work is the norm, or at least close to being the norm. (And, as you say, good work is often rejected.) > Its painful to me that so much of the acceptance process for research papers (in my field at least) is based on luck. I don't think it's based on luck, it's based on politics. Academic science is a racket. There is some luck too, though.
- seccess 12y ago
  > Slips through? Mediocre work is the norm, or at least close to being the norm. In computer science, this is likely because of the ridiculous number of conferences there are and how eager each one is to fill out their roster. I want to hear of the conference that published no papers this year because none were good enough for publication. > I don't think it's based on luck, it's based on politics. Academic science is a racket. I've heard (more than once) of people who submit a paper, have it rejected, and then get scooped by a professor who was on the PC of the conference that rejected them. There is a very real possibility of work being accepted or rejected purely because of what the reviewer thinks will be most beneficial to their career. So, I think your cynicism is well-placed.
  - javert 12y ago
    > In computer science, this is likely because of the ridiculous number of conferences there are and how eager each one is to fill out their roster. I want to hear of the conference that published no papers this year because none were good enough for publication. Yep. But there is a reason it is this way, which is that to be successul on the academic feeding trough/career path, you have to serve on PC committees and you have to publish a lot of papers. > There is a very real possibility of work being accepted or rejected purely because of what the reviewer thinks will be most beneficial to their career. I think this is right, but it's less often so work can be scooped, and more often about building up your "camp." As a research professor, you need your area of expertise to be popular and you need to get a lot of citations for your work. You want to be a leader in an area that actually matters, where your definition of "actually matters" is actually purely herd mentality: what do other people consider to be important? So you want to reject work that bolsters your camp, cites you, etc., and reject work that poses a threat to the ascendancy of your area of expertise.
    - blueMist 12y ago
      Agree with a lot of what is being said here. Have seen vicious anonymous reviews in CS - as we all have - both due to someone misunderstanding the work but also due to people from "a different camp" simply disliking a given approach. This must be common in other fields as well. Someone said on this thread that anonymity is not a guarantee of quality - completely agree. The move towards "open reviewing" for conferences in CS is very encouraging, however we will see how far it gets.
      - javert 12y ago
        I'm going to say something controversial. Minor reform of the referee system is treating the symptoms, not the disease. The disease is that there is no honest, mutual, voluntary exchange between a party that values a given research project and the party that produces it. Rather, we have federal bureaucrats handing out money taken from taxpayers willy-nilly via an old-boy network as described above. (The difference between taxpayers and slaves is that taxpayers only must sacrifice a fraction of their productivity to this absurd system, not all of it.) In CS, all the research we do either goes to benefit shareholders of companies that ultimately profit from it, or (the majority) is just ignored because it's part of the paper mill competition. Those companies should be the ones paying for the research, not the taxpayers. Those companies are free riding, and "we the people" should put an end to it.
tehwalrus 12y ago
We ended up with 5 peer reviewers and an Editor override for one of our papers, instead of the usual 2 (I was the PhD student who actually did the algebra, so I saw all the comments.) In fact, even after so much work to get it published, we now know of two errors in it - neither were pointed out by the reviewers of course. I can't help thinking that that one experience sums up all of your problem cases above :)
ehurrell 12y ago
I've had similar experience, and it can be tough, especially when the lack of interest from a reviewer is obvious. I still joke about the review I got that included the line "English need improving".
deleted 12y ago
[deleted]

by seccess 12y ago

javert 12y ago

I'm also a computer science PhD student. I don't disagree with anything you said, but I am even more cynical. > Of course, the opposite can happen where mediocre works slips through Slips through? Mediocre work is the norm, or at least close to being the norm. (And, as you say, good work is often rejected.) > Its painful to me that so much of the acceptance process for research papers (in my field at least) is based on luck. I don't think it's based on luck, it's based on politics. Academic science is a racket. There is some luck too, though.

seccess 12y ago
> Slips through? Mediocre work is the norm, or at least close to being the norm. In computer science, this is likely because of the ridiculous number of conferences there are and how eager each one is to fill out their roster. I want to hear of the conference that published no papers this year because none were good enough for publication. > I don't think it's based on luck, it's based on politics. Academic science is a racket. I've heard (more than once) of people who submit a paper, have it rejected, and then get scooped by a professor who was on the PC of the conference that rejected them. There is a very real possibility of work being accepted or rejected purely because of what the reviewer thinks will be most beneficial to their career. So, I think your cynicism is well-placed.
- javert 12y ago
  > In computer science, this is likely because of the ridiculous number of conferences there are and how eager each one is to fill out their roster. I want to hear of the conference that published no papers this year because none were good enough for publication. Yep. But there is a reason it is this way, which is that to be successul on the academic feeding trough/career path, you have to serve on PC committees and you have to publish a lot of papers. > There is a very real possibility of work being accepted or rejected purely because of what the reviewer thinks will be most beneficial to their career. I think this is right, but it's less often so work can be scooped, and more often about building up your "camp." As a research professor, you need your area of expertise to be popular and you need to get a lot of citations for your work. You want to be a leader in an area that actually matters, where your definition of "actually matters" is actually purely herd mentality: what do other people consider to be important? So you want to reject work that bolsters your camp, cites you, etc., and reject work that poses a threat to the ascendancy of your area of expertise.
  - blueMist 12y ago
    Agree with a lot of what is being said here. Have seen vicious anonymous reviews in CS - as we all have - both due to someone misunderstanding the work but also due to people from "a different camp" simply disliking a given approach. This must be common in other fields as well. Someone said on this thread that anonymity is not a guarantee of quality - completely agree. The move towards "open reviewing" for conferences in CS is very encouraging, however we will see how far it gets.
    - javert 12y ago
      I'm going to say something controversial. Minor reform of the referee system is treating the symptoms, not the disease. The disease is that there is no honest, mutual, voluntary exchange between a party that values a given research project and the party that produces it. Rather, we have federal bureaucrats handing out money taken from taxpayers willy-nilly via an old-boy network as described above. (The difference between taxpayers and slaves is that taxpayers only must sacrifice a fraction of their productivity to this absurd system, not all of it.) In CS, all the research we do either goes to benefit shareholders of companies that ultimately profit from it, or (the majority) is just ignored because it's part of the paper mill competition. Those companies should be the ones paying for the research, not the taxpayers. Those companies are free riding, and "we the people" should put an end to it.

tehwalrus 12y ago

We ended up with 5 peer reviewers and an Editor override for one of our papers, instead of the usual 2 (I was the PhD student who actually did the algebra, so I saw all the comments.) In fact, even after so much work to get it published, we now know of two errors in it - neither were pointed out by the reviewers of course. I can't help thinking that that one experience sums up all of your problem cases above :)

ehurrell 12y ago

I've had similar experience, and it can be tough, especially when the lack of interest from a reviewer is obvious. I still joke about the review I got that included the line "English need improving".

deleted 12y ago

[deleted]