6 ms·

Why would someone do this?

by OedipusRex 12y ago

Blackthorn 12y ago
Publication is a high-stakes game for academics.
- OedipusRex 12y ago
  I have heard that. If I remember correctly many colleges push for a certain number of publications per year.
  - superuser2 12y ago
    It's pretty difficult to lose tenure (that's the point), but publication volume and quality are pretty much the metric for hiring and promoting faculty. Just like standardized testing at the undergraduate admissions level, it's pretty much inevitable that some people will attempt to game the metric instead of what it's supposed to reflect.
    - scott_s 12y ago
      I think that hiring faculty will tend to be primarily about publications. But promoting faculty will also depend on pulling in grants - perhaps even more so.
      - p4bl0 12y ago
        True. But guess what is among the important parameters to decide to whom a grant is given. Bingo: bibliometrics here too.
        scott_s 12y ago
        Of course. But if a professor has the publications, but not the grants, they may not get promoted.
      - jessaustin 12y ago
        It's my understanding that grants themselves go more to those with more publications and citations.
        Tloewald 12y ago
        Grants and publications become a virtuous circle. Getting grants gives you time and resources (and prestige) to get publications. Getting publications is the gateway to getting grants (before you have grants), and getting more grants (once you have grants). E.g. the measure of a grant-getter's success (at utilizing a grant) is generally the number and "quality" of publications the grant generates.
        scott_s 12y ago
        I agree, but you're leaving out the money part. Schools get a cut of all grants. From the school's perspective, the grant itself is valuable. So when it comes time to evaluate tenure and other promotions, they will look at both.
        deleted 12y ago
        [deleted]
        pgbovine 12y ago
        yep, absolutely right. the amounts vary a lot by school, but typically your school gets 1/3. So if your grant is for $300k, you keep $200k, and your school gets $100k. thus when evaluating two faculty with comparable numbers of publications, the one who's brought in more money is more valuable to the department ... literally :)
    - Estragon 12y ago
      I would say the metric, at least in fields with a lot of money, is actually the amount of funding you bring in. Of course, publication record has a big influence on that, too.
  - jboggan 12y ago
    This trickles down to the grad students and their relationship with their advisors. The deal we were given was: "publish three papers and you get your PhD". This creates some really perverse incentives on both sides.
    - pgbovine 12y ago
      that sounds like pretty clear expectations to me, much better than the nebulous "do something novel for a few years, impress some old people, and then maybe you'll get a PhD, just maybe." yes, the game is imperfect and always stacked against newbies (as any game is), but at least you know where the goalposts are. the experience of my peers in grad school mirrored this "three papers -> PhD" heuristic: http://www.pgbovine.net/PhD-overview.htm http://www.pgbovine.net/PhD-overview.htm
Pxtl 12y ago
The most common numerical metric for the quality of a paper is the number of times it is cited.
- mdda 12y ago
  I've heard that often in 'blind' peer reviews, the feedback will often be : "Paper would be improved by including reference to the work of XYZ" - essentially creating additional citations for author XYZ (who may, coincidentally, be the reviewing peer, and may feel that getting additional citations for their own work is the payment they require for doing the review work).
  - aganders3 12y ago
    This definitely happens, but often you're asked to review a paper _because_ of your experience or publication record in that area. If someone neglects to cite your related work, should you just let that slide?
    - Chronic29 12y ago
      What if your work sucks and doesn't deserve to be cited? Sorry but a researcher can't cite every single paper about their topic.
p4bl0 12y ago
Because of bibliometrics. Academics are judged on stupid factors depending on the quantity of publications and citations rather than on the quality of their research. The idea of bibliometrics is that you can compare on a single dimension individuals who do not work on the same subject, who use different methods, who do it in different communities which have different publication habits, etc. Of course this is completely stupid. It's like saying that a cyclist who has participated in 10 Tour de France is better than a judoka who has participated to only 3 Olympic games. Only here you also give a job and funding to do it appropriately to the cyclist and say to the judoka to go f* himself until he participates in more Tour de France. Of course, the thing is that you can't even have good criteria, because necessarily if you give criteria they become the goal, and instead of doing good research and publishing what is necessary when and where it makes sense, academics are forced to do research and have a publication policy that satisfy the arbitrary criteria. And even if the unlikely case where the criteria match what's better, it is impossible for it to be the case in the so many different fields of research. The only good solution would be to judge academics by the two or three most relevant publications they have on the matter of the grant or job. That would require reading the publications, and if possible the report of the peer-review process (which would be enabled by open peer review).
- FatalLogic 12y ago
  It is a dire situation, but the fact that the researcher(s) went to great effort to get into that particular journal shows that at least the perceived quality of the journal is still being used as a rough metric for the quality of the research. So quality is still a factor, even if quantity has far too much weight. If they had simply wanted to get published, then it seems like there are plenty of journals that will accept almost anything (they get paid on publication). More than 150 out of 300 open access journals accepted an obvious spoof paper last year[1]. [1]http://www.sciencemag.org/content/342/6154/60.full http://www.sciencemag.org/content/342/6154/60.full
  - p4bl0 12y ago
    Of course, the "impact factor" of the journal also counts, however it is gamed too. For instance there are publishers which artificially increase the impact factors of their journals by requesting that authors of accepted papers cite a few articles recently published in their journal. Now, I would like to come back to the "article" you cite. Let me get this straight from the beginning: this text (by John Bohannon) is a piece of shit. Let me explain why: The article aim to compare open access journals and closed access ones. The method that is used for that purpose is quite remarkable. It consists in sending a paper of very poor quality with wrong results in it to many gold open access journals (this means that there are what is called "Author Processing Charge": the authors pay the journal to publish), and to see how many of those will accept the paper. What happens is that a bit more than half of them accept the paper with, of course, not a single sign of peer-review happening. The author then conclude that open access journals are for the major part of poor quality, implying that they are worse than "traditional journals". This conclusion is eminently ridiculous. First, if you want to compare open access and closed access journals, you also need to actually test the closed access journals. You can't just assume that they are good, especially when it's implied and not a single reason to think so is given (trust me it could be hard in many cases). Second, the method makes no sense. No researchers sends his or her paper to unknown journals. When a paper is submitted to a journal, the journal is chosen depending on at least two criteria: on one side its prestige, because the more prestigious, the more read, and the goal of an academics is of course to be read by the other researchers in his domain; on the other side, for the seriousness of the journal (which is tacitly known in its academic community), because the peer-review process is very important for the authors (no academics like to have a paper published and then discover that there is a mistake in it). Amusingly, this is something that scientific news magazine such as Nature and Science are not that good at. Third, the real conclusion of the poorly carried experiment that the author makes is that the bibliometrics pressure on researcher (“publish or perish”) coupled to their desirable and natural wish to be more widely read, and thus to publish in open access journals, gave birth to an unhealthy publication business that is harmful to research and science (I recall that the author only experimented with authors-pay open access journal, calling it simple "open access", as if he was, strangely, paid by publishers lobby to increase the already existing confusion between gold open access (where authors pay) and open access, which exists in many different models). To me, the real solution to all these problem lies in Diamond Open Access and Open Peer-Review.
ps2fats 12y ago
In some countries, you get paid to publish in scientific journals. It is a way for the country to get recognition. There is massive incentive then to publish, apart from career incentives.

Why would someone do this?

Blackthorn 12y ago

Publication is a high-stakes game for academics.

OedipusRex 12y ago
I have heard that. If I remember correctly many colleges push for a certain number of publications per year.
- superuser2 12y ago
  It's pretty difficult to lose tenure (that's the point), but publication volume and quality are pretty much the metric for hiring and promoting faculty. Just like standardized testing at the undergraduate admissions level, it's pretty much inevitable that some people will attempt to game the metric instead of what it's supposed to reflect.
  - scott_s 12y ago
    I think that hiring faculty will tend to be primarily about publications. But promoting faculty will also depend on pulling in grants - perhaps even more so.
    - p4bl0 12y ago
      True. But guess what is among the important parameters to decide to whom a grant is given. Bingo: bibliometrics here too.
      - scott_s 12y ago
        Of course. But if a professor has the publications, but not the grants, they may not get promoted.
    - jessaustin 12y ago
      It's my understanding that grants themselves go more to those with more publications and citations.
      - Tloewald 12y ago
        Grants and publications become a virtuous circle. Getting grants gives you time and resources (and prestige) to get publications. Getting publications is the gateway to getting grants (before you have grants), and getting more grants (once you have grants). E.g. the measure of a grant-getter's success (at utilizing a grant) is generally the number and "quality" of publications the grant generates.
        scott_s 12y ago
        I agree, but you're leaving out the money part. Schools get a cut of all grants. From the school's perspective, the grant itself is valuable. So when it comes time to evaluate tenure and other promotions, they will look at both.
        deleted 12y ago
        [deleted]
        pgbovine 12y ago
        yep, absolutely right. the amounts vary a lot by school, but typically your school gets 1/3. So if your grant is for $300k, you keep $200k, and your school gets $100k. thus when evaluating two faculty with comparable numbers of publications, the one who's brought in more money is more valuable to the department ... literally :)
  - Estragon 12y ago
    I would say the metric, at least in fields with a lot of money, is actually the amount of funding you bring in. Of course, publication record has a big influence on that, too.
- jboggan 12y ago
  This trickles down to the grad students and their relationship with their advisors. The deal we were given was: "publish three papers and you get your PhD". This creates some really perverse incentives on both sides.
  - pgbovine 12y ago
    that sounds like pretty clear expectations to me, much better than the nebulous "do something novel for a few years, impress some old people, and then maybe you'll get a PhD, just maybe." yes, the game is imperfect and always stacked against newbies (as any game is), but at least you know where the goalposts are. the experience of my peers in grad school mirrored this "three papers -> PhD" heuristic: http://www.pgbovine.net/PhD-overview.htm http://www.pgbovine.net/PhD-overview.htm

Pxtl 12y ago

The most common numerical metric for the quality of a paper is the number of times it is cited.

mdda 12y ago
I've heard that often in 'blind' peer reviews, the feedback will often be : "Paper would be improved by including reference to the work of XYZ" - essentially creating additional citations for author XYZ (who may, coincidentally, be the reviewing peer, and may feel that getting additional citations for their own work is the payment they require for doing the review work).
- aganders3 12y ago
  This definitely happens, but often you're asked to review a paper _because_ of your experience or publication record in that area. If someone neglects to cite your related work, should you just let that slide?
  - Chronic29 12y ago
    What if your work sucks and doesn't deserve to be cited? Sorry but a researcher can't cite every single paper about their topic.

p4bl0 12y ago

Because of bibliometrics. Academics are judged on stupid factors depending on the quantity of publications and citations rather than on the quality of their research. The idea of bibliometrics is that you can compare on a single dimension individuals who do not work on the same subject, who use different methods, who do it in different communities which have different publication habits, etc. Of course this is completely stupid. It's like saying that a cyclist who has participated in 10 Tour de France is better than a judoka who has participated to only 3 Olympic games. Only here you also give a job and funding to do it appropriately to the cyclist and say to the judoka to go f* himself until he participates in more Tour de France. Of course, the thing is that you can't even have good criteria, because necessarily if you give criteria they become the goal, and instead of doing good research and publishing what is necessary when and where it makes sense, academics are forced to do research and have a publication policy that satisfy the arbitrary criteria. And even if the unlikely case where the criteria match what's better, it is impossible for it to be the case in the so many different fields of research. The only good solution would be to judge academics by the two or three most relevant publications they have on the matter of the grant or job. That would require reading the publications, and if possible the report of the peer-review process (which would be enabled by open peer review).

FatalLogic 12y ago
It is a dire situation, but the fact that the researcher(s) went to great effort to get into that particular journal shows that at least the perceived quality of the journal is still being used as a rough metric for the quality of the research. So quality is still a factor, even if quantity has far too much weight. If they had simply wanted to get published, then it seems like there are plenty of journals that will accept almost anything (they get paid on publication). More than 150 out of 300 open access journals accepted an obvious spoof paper last year[1]. [1]http://www.sciencemag.org/content/342/6154/60.full http://www.sciencemag.org/content/342/6154/60.full
- p4bl0 12y ago
  Of course, the "impact factor" of the journal also counts, however it is gamed too. For instance there are publishers which artificially increase the impact factors of their journals by requesting that authors of accepted papers cite a few articles recently published in their journal. Now, I would like to come back to the "article" you cite. Let me get this straight from the beginning: this text (by John Bohannon) is a piece of shit. Let me explain why: The article aim to compare open access journals and closed access ones. The method that is used for that purpose is quite remarkable. It consists in sending a paper of very poor quality with wrong results in it to many gold open access journals (this means that there are what is called "Author Processing Charge": the authors pay the journal to publish), and to see how many of those will accept the paper. What happens is that a bit more than half of them accept the paper with, of course, not a single sign of peer-review happening. The author then conclude that open access journals are for the major part of poor quality, implying that they are worse than "traditional journals". This conclusion is eminently ridiculous. First, if you want to compare open access and closed access journals, you also need to actually test the closed access journals. You can't just assume that they are good, especially when it's implied and not a single reason to think so is given (trust me it could be hard in many cases). Second, the method makes no sense. No researchers sends his or her paper to unknown journals. When a paper is submitted to a journal, the journal is chosen depending on at least two criteria: on one side its prestige, because the more prestigious, the more read, and the goal of an academics is of course to be read by the other researchers in his domain; on the other side, for the seriousness of the journal (which is tacitly known in its academic community), because the peer-review process is very important for the authors (no academics like to have a paper published and then discover that there is a mistake in it). Amusingly, this is something that scientific news magazine such as Nature and Science are not that good at. Third, the real conclusion of the poorly carried experiment that the author makes is that the bibliometrics pressure on researcher (“publish or perish”) coupled to their desirable and natural wish to be more widely read, and thus to publish in open access journals, gave birth to an unhealthy publication business that is harmful to research and science (I recall that the author only experimented with authors-pay open access journal, calling it simple "open access", as if he was, strangely, paid by publishers lobby to increase the already existing confusion between gold open access (where authors pay) and open access, which exists in many different models). To me, the real solution to all these problem lies in Diamond Open Access and Open Peer-Review.

ps2fats 12y ago

In some countries, you get paid to publish in scientific journals. It is a way for the country to get recognition. There is massive incentive then to publish, apart from career incentives.