14 ms·

Among top researchers 10% publish at unrealistic levels, analysis finds

188 points by crescit_eundo 2y ago

crescit_eundo 2y ago
https://archive.ph/SnYjB https://archive.ph/SnYjB
seydor 2y ago
note that the 2% list is very large, thus most of the scientists are not just the leading edge superstars but also run of the mill scientists and principal investigators. The authors focus on the citation inflation in younger scientists but that s unfair (it's only 1000 out of the 20000 in the top 10%). The reality is that older established scientists are much more advantaged because the people of their lab are sought after as collaborators , and they autonatically get an authorship as last authors by virtue of being principal investigators, even if they don't even take a look at the paper. It's thus not strange that they get 35+ authorships per year
- johnea 2y ago
  This is pretty much what I came here to post. Anyone who's even been adjacent to scientific academia knows that the lead author is typically the advisor to the grad student or post doc that actually did the research and wrote the paper.
  - asdff 2y ago
    In my experience the lead author is always the post doc or grad student who actually did the research and wrote the paper, while the advisor is the last author and they developed the grant to fund both the grad student and post doc. If you collaborate with another lab, their post docs go towards the front behind your name and the advisor goes somewhere at the end. If two people equally wrote a paper they might be 1 and 2 and get an asterisk indicating they are both co first authors. If anyone was involved at all in the work e.g data collection or anyone involved with running an assay they go somewhere in the middle. Perhaps it’s different in other fields
    - buildbot 2y ago
      Yep, this is what I learned in Grad school as well. First author is the primary person doing the work typically, then their advisor will be the last, and then you work towards the middle. So names in the middle of a paper typically contribute less than either edge!
    - jltsiren 2y ago
      This sounds like a terminological issue. Without further context, I would assume that "lead author" means the last/senior author, not the first/junior author. But I guess that's a life sciences tradition. "Who actually did the research" is not always an accurate description of the first author. There are plenty of papers, where the last author had already contributed enough to justify authorship before the first author was even hired. You often need to develop the idea and get preliminary results to convince someone to fund the project, before you get the money to hire the first author to work on the idea.
    - danielthor 2y ago
      When I was a grad student the only time my PI was listed first on a paper was when he actually did most of the work and I just helped with maths. Otherwise the PI was last and the grad students and post docs doing the research up front. People doing data collection or other ancillary work were mentioned in the acknowledgement section at the end. When the author count is much higher, like in the thousands in high energy physics, there are different norms like listing the authors alphabetically if the paper summarizes many significant achievements. In the in-between author order is negotiated.
  - levocardia 2y ago
    ...and imagine how well it would go over for a grad student to say "actually Dr Advisor, you don't meet the criteria for authorship"
    - throwawaymaths 2y ago
      I've seen someone who desperately wanted their nobel laureate PI on their paper but the professor did not put his name on the paper because it was all their idea (even though it was his grant, he reviewed and ok'd the paper, paper was relevant to his lab, and he really spoke effusively at their thesis defense)
  - zevon 2y ago
    This varies greatly by discipline and tradition - sometimes even within the same research groups. For example, some people put the senior person / advisor first, some put this person last, some go by alphabet. In my experience, there is even more variation in the amount of work expected to be put into a publication by said senior person. To put it mildly. ^^
- mvdtnz 2y ago
  These researchers compared the publish rates of Stanford Top 2% scientists with comparable Nobel laureates and the rate of publishing among the anomalous researchers was ludicrously higher than their peers at the top of their game - "up to 212 publications per year, and 5792 new coauthors per year, compared to up to 28.3 publications per year and 173 new coauthors per year on NL [nobel laureates]". Do you have any reason to believe the T2 group is composed of "older established scientists" who are "much more advantaged" than the group of Nobel Laureates? To the tune of 10x the number of publications and 33x coauthorships?
  - seydor 2y ago
    > reason to believe the T2 group is composed of "older established scientists" there were 1000 academics with < 10 years of career and 19000 with more (in the inflators list). I did not compare their advantage to the nobel scientists but to the younger ones. (Also the nobel laureates are extreme outliers who don't need any more citations to satisfy their ego or funding needs)
- dleeftink 2y ago
  It's interesting how the observers of this preferential attachment effect have since become, well, very preferentially attached [0]. The scaling and diffusion of ideas is rarely (and unfortunately), equally distributed. [0]: https://scholar.google.com/scholar?q=Emergence+of+scaling+in+random+networks https://scholar.google.com/scholar?q=Emergence+of+scaling+in...
- fph 2y ago
  Authorship norms in certain fields seem completely broken: a PI shouldn't become an author just for providing money and a lab. Yet unfortunately that's the norm in many fields.
  - fooker 2y ago
    > a PI shouldn't become an author just for providing money and a lab. What have the romans ever given us? https://www.youtube.com/watch?v=Qc7HmhrgTuQ https://www.youtube.com/watch?v=Qc7HmhrgTuQ
    - fph 2y ago
      I'm not claiming the money and lab are useless, but just don't call them "authors". When one publishes something with a grant from the NIH or the European Union or the Bill Gates foundation, they acknowledge the grant, they don't add the NIH or Bill Gates as an author. How is this different?
amelius 2y ago
The 10x scientists.
- marginalia_nu 2y ago
  To be fair, there has always been extreme outliers in scientific output, like Gauss and Euler.
LudwigNagasena 2y ago
> In our opinion, it is difficult to believe that the majority of authors with excessive rates are able to consistently produce large quantities of high quality or groundbreaking research, with their input to each paper being substantial and within norms of what constitutes co-authorship Huh, I thought providing the general direction and sanity-checking the results is the norm of what constitutes co-authorship for a top professor.
- mvdtnz 2y ago
  Do you think any one person can provide direction and sanity-check the work of more than 5,000 papers in a year? I don't.
  - thinkingtoilet 2y ago
    Where did you get the 5000 number?
    - mvdtnz 2y ago
      From the research paper that the article is discussing. https://www.tandfonline.com/doi/full/10.1080/08989621.2024.2445280 https://www.tandfonline.com/doi/full/10.1080/08989621.2024.2... > Namely, rates on T2 of up to 212 publications per year, and 5792 new coauthors per year, compared to up to 28.3 publications per year and 173 new coauthors per year on NL. This is despite that the NL list is weighted with more senior researchers, often at the pinnacle of their careers – and hence – likely to have higher rates.
      - LPisGood 2y ago
        That is 212 papers, not 5000. 5000 is the number of new authors of those 200 papers.
        Xelynega 2y ago
        Looks like the number is off, but is their point any different with 212 papers? I still think that it would be impossible to give any meaningful support to 212 papers in a year. That's a new paper every 1.72 days.
irrational 2y ago
So, who is the worst? Is there someone pumping out multiple new papers per day?
- cgh 2y ago
  From the article: "Similarly, paper mills – organizations that produce fabricated or low-quality manuscripts for profit – often rely on inflated co-authorship networks to boost the metrics of paying researchers..." The article references this paper on predatory journals as the source: https://link.springer.com/article/10.1186/s12916-015-0469-2?utm_source=getftr&utm_medium=getftr&utm_campaign=getftr_pilot&getft_integrator=tfo https://link.springer.com/article/10.1186/s12916-015-0469-2?... According to this paper more than 75% of the authors published in these journals originate in India and Africa (mostly Nigeria).
zzzeek 2y ago
[flagged]
- UncleMeat 2y ago
  Firing a shitload of people at random makes it more difficult to root out these things, not less.
- rlili 2y ago
  cool, will it do anything about CEOs which earn 100x what a common employee earns, despite not outputting 100x?
  - kibwen 2y ago
    Happily, yes. They will now earn 10,000x what a common employee earns while doing less than ever before.
  - BeetleB 2y ago
    In many (traditional) companies, a CEO's mistake can easily cause 100x more harm than an average worker's mistake.
    - nosianu 2y ago
      And when that happens, what happens to them? There "risk" is to get a lot of "golden parachute" money to leave earlier than their contract says, as a maximum penalty. If there us actual money to pay the company or insurance does it for them. . The human network of processes, laws, culture, knowledge established over millennia, which enables those CEOs to be at the top of such valuable companies in the first place was built by billions over time. Those CEOs are not bette than others, or people of the past, they are the same humans. That they now move billions instead of having a small stone axe business for the local tribes is not of their making. They were lucky. If you want to see someone really earning the hundred million with their own hard work, place them in the middle of nowhere, and with no access to the global human network that they did nothing whatsoever to construct because it was already there. The value that CEOs and others grab for themselves is not of their making. They are able to have such huge businesses only because a human network in space (the current one) and time (handed forward over time, knowledge and processes and tools). They "deserve" the millions just as much as everybody else. They misuse their superior position in the network, since they sit in places where lots of value not of their own making is going through. In addition to that, in many important cases they managed to catch all the upside, but let all the failures fall through to be caught be everybody else. I have nothing whatsoever against true entrepreneurs. I know a few personally! But as you reach the billions, and the top management (instead of direct owners who actually suffer when the company fails to deliver), the justification for the incomes gets more tenuous IMO. That is one of my big issues with the "Left" - here in Germany their new program includes getting rid of billionaires. It's too simple and too one-dimensional. The diversity of people in such high-value positions is huge, and there are many more dimensions! The left treat it like a one-dimensional scale, starting with a normal person that has a few dollars or Euros, and on that scale and idea of "money" they remain with their arguments. Instead, we have to look at the many many different things all that money actually stands for. CEO income with their usually much reduced risk compared to owners should not be treated the same, for example and IMHO.
readyplayernull 2y ago
Among given population X% are suspicious and Y% keep an eye on statistic and their heads below the radar.
recursivedoubts 2y ago
Academia is filled to the absolute brim with scams, news at 11. (I'm in academia)
- gjsman-1000 2y ago
  Yet they still demand we take them with utmost seriousness...
  - pazimzadeh 2y ago
    not really, maybe you're thinking of medical doctors
- matthewdgreen 2y ago
  All human activity is full of scams if you cast your net wide enough. Insofar as academia contains scams, the distinction is that they have a much lower profit margin.
  - cwmoore 2y ago
    How does one compute the profit margin of academia?
    - matthewdgreen 2y ago
      You look at the bad actors (in this case and others) and see how much money or equivalent they're getting out of their work. My general view is that it's much less than you could obtain with a PhD and similar effort applied to other fields.
  - cherryteastain 2y ago
    > Sayre's law states, in a formulation quoted by Charles Philip Issawi: "In any dispute the intensity of feeling is inversely proportional to the value of the issues at stake." By way of corollary, it adds: "That is why academic politics are so bitter." https://en.wikipedia.org/wiki/Sayre%27s_law https://en.wikipedia.org/wiki/Sayre%27s_law
TrackerFF 2y ago
If you scrutinize the top publishers, I'm sure you'll find some duplicate publications, salami slicing, and other things. When you're that prolific, the number of publications also become a goal, especially if you're awarded for it. Maybe not outright academic fraud, but at least publishing pattern that tries to maximize some measure(s).
YossarianFrPrez 2y ago
I was just thinking about the upper bound on how many papers one could publish in a year earlier this morning. Speaking as someone who spent years in Tech and is now in the middle of a Ph.D. program... Within the current paradigm, where a post-doc gets hired as a new professor and goes about starting the rough equivalent of a single private sector team, at least in my subfield, 15-25 (non-first author publications) a year is an impressive number. And thus the numbers cited in the supplemental materials, the max being 136 papers a year, is strange, and I am pretty sure the author's points about paper mills etc. hold true. This is the great thing about the "Contributor Roles Taxonomy" system: it provides a lower level of abstraction and gives credit for who did what (idea generation, coding, writing, reviewing, raising the money, etc.) compared with using "a publication" at the unit of measure. It really solves a lot of problems. [1] [1] https://authorservices.wiley.com/author-resources/Journal-Authors/open-access/credit.html https://authorservices.wiley.com/author-resources/Journal-Au... But I'd also like to raise another point. People who wind up in Academia tend to go straight from undergrad to grad school (or spend a year being a lab manager in academia) and so most if not all of the systems we use in software development aren't present. Code review, project timeline estimation, building up a lab-wide codebase of functions to speed up repetitive tasks, 360 degree reviews, lab-wide project management software, an org-chart deeper than two (or in rare cases, three) layers, teams with differentiated responsibilities multiple teams, etc. are not the norm. Every so often I hear of one lab here or there that does one or two of these, not all of them. (Though my experience is limited.) My point is that if one were to apply all of the modern systems used for coordinating groups of people to produce structured forms of writing, etc., then 100 papers a year sans a breadth-vs.-depth tradeoff might just be doable. But note that this is not "100 papers as year" from an individual, it's "100 papers a year from a mid-sized institution." (Six teams of four getting out 1.5 papers a month equates to 108 papers a year, near the maximum cited above.) Bell Labs' publication / patent rate must have been high! Granted, what I'm saying is not exactly within the current paradigm of how science is done, and might not be possible in a university setting.
Steuard 2y ago
It's quite possible that there's a real effect here. But while I've only had time to skim parts of the paper, I don't see any indication of whether the authors have accounted for the different norms in different fields when analyzing their data for potentially fraudulent or deceptive behavior. Just for example, physics papers produced by large international collaborations (e.g. every single paper from the Large Hadron Collider at CERN) routinely have hundreds of authors (e.g. https://www.nature.com/articles/nature.2015.17567 https://www.nature.com/articles/nature.2015.17567): everyone who has made substantial contributions to the design and operation of the facility is listed, as is everyone on the data analysis teams. (My understanding is that people in those specific fields all recognize that "number of citations" is a mostly meaningless number for those involved, and other metrics for productivity are well-known in those communities and routinely used.) I hear that some genomics papers have broken 1000 authors as well. I could easily imagine that the high end of observed publication numbers and coauthor counts would be dominated by those giant collaborations, even though there is absolutely no attempt to mislead anyone in the process. Can anyone tell from this article to what degree its conclusions might be influenced by this factor?
daft_pink 2y ago
We’re going to penalize people for publishing too mucn now? Really? Just want to say that I love Issac Asimov and he wrote a lot of high quality work and people even say that he’s a graphomaniac with a writing compulsion. I think people’s work should be judged by the output quality not some sort of speed limiting judgment.
- kergonath 2y ago
  > Just want to say that I love Issac Asimov and he wrote a lot of high quality work and people even say that he’s a graphomaniac with a writing compulsion. There is a difference between being prolific and what those people put out. At least one order of magnitude. Isaac Asimov did not write 80 novels each year.
- Xelynega 2y ago
  Isaac Asimov wroted/edited ~500 books over his life. Some of the people mentioned in the research are coauthoring ~200 paper a year. I don't think Isaac would be in the same league as those mentioned in this paper.
- robertlagrant 2y ago
  Isaac Asimov's work was judged by how many people published it, and how many people bought it. There's no equivalent quality control for these papers; it's much more the honour system.
- staticman2 2y ago
  This is more like criticizing Asimov for allowing a magazine to be published called "Isaac Asimov's Science Fiction Magazine" where he was not the editor and it was not his magazine.
userbinator 2y ago
I wonder how many of these papers are now mostly if not completely AI-generated. This happened a year ago, and only gained attention because it was bad enough to be noticed: https://news.ycombinator.com/item?id=39391034 https://news.ycombinator.com/item?id=39391034
alberth 2y ago
Isn't this a well-known issue in the academic and research community where researchers include their friends' names on their published work? Since researchers are rated and measured by the number of published papers they have, many people game the system by exchanging bylines with their friends ... so that it boosts their total number of papers published.
- mrweasel 2y ago
  This was one of the ethical dilemmas we had to go through at university some 20 years ago: "Your professor/supervisor tells you to add them to the list of authors on your paper, do you do it?". There's a whole subset of "issues" here, one where you're more likely to be published with the supervisors name on the paper, another where you're travelling to present your paper, but you're not likely to be granted the budget and adding a professors name will almost ensure an all expenses paid trip. Low hundreds papers doesn't seem impossible, nor does hundreds of collaborators, but it would heavily depend on your work. We see the same in software. Some people seems unrealistically productive, spawning one successful project after another. I would be interested in knowing for how long they can keep up the pace though. There's also the question of the quality of their work.
thinkingQueen 2y ago
My eyes were opened when a field called gamification appeared in the early 2010s. In a few years many gamification researchers had tens of thousands of citations, h-indexes nearing hundred. Well, if you think about it they’re gamers, they’ve been grinding their RPG characters, sniping skills and whatnot for thousands of hours. It’s only natural that these guys and girls figure out how to reach the maximum scientific high score. Some of the gamification researchers are near the top 500 of that 2% list. Now ask yourself, is gamification something that should make you one of the top 500 scientist in the world? I doubt it, but modern science is a citation game. Nothing else.
- hansvm 2y ago
  That feels like a certain xkcd [0]. If you can game the totally bullshit rankings then clearly your papers are worth something. [0] https://xkcd.com/2385/ https://xkcd.com/2385/
  - lmm 2y ago
    Thought that was going to be https://xkcd.com/125/ https://xkcd.com/125/. Iteration of the same concept I guess.
- xdavidliu 2y ago
  reminds me of the fraudulent Ariely and Gino papers that were exposed a year or two ago, where a very common comment was "they're dishonesty researchers, of course they are going to be dishonest".
- layman51 2y ago
  Gamification as a field of study also reminds me a lot of this video[1] of a talk about tips for game developers. It is very interesting because it is from 2016 which is close to the time period you mentioned. I’m pretty sure this video has been shared on Hacker News a bunch. [1]: https://www.youtube.com/watch?v=xNjI03CGkb4 https://www.youtube.com/watch?v=xNjI03CGkb4
godelski 2y ago
It amazes me how many "high profile" researchers have blatantly fraudulent work. As an example, take this rather recent ICLR submission that is egregiously plagiarized[0] (I mean attempts to hide the plagiarism are minimal to nonexistent). Clicking on the authors you can view their profiles and you'll find that most authors have several thousand citations and one north of 25k[1]!!!! They also have an i-10 of 400, which is highly abnormal for that citation level while the last author has an i-10 of 126 with only 7.3k citations. The problem I see here is that there's lots of suspicious activity and clear evidence of fraud, yet little to no action is taken. If anything, it seems we mostly ignore it. The ICLR submission is a bit odd too, since decisions and comments are public, most of the time this happens behind closed doors and we don't observe the evidence. As far as I'm aware, no one is making network graphs to track fraudulent authors. I think we want to sweep it under the rug because at the end of the day our system relies heavily upon trust. Unless we strongly incentivize reproduction we require high trust and we're afraid to acknowledge fraud in the community because we believe it will make more distrust science. But I think in reality, the reluctance to pursue fraud and the reluctance to have high clarity creates more distrust. The truth is that this system poorly scales even before we incorporate the reverse (bureaucratic) incentives. I think we lost sight of the goals of research and publishing. Publishing exists to facilitate communication among our peers. We publish in venues for two reasons: to improve/error check works and to facilitate social networks. Publish or perish is detrimental as well as the push for highly subjective measures like novelty. I strongly advocate for returning to the old paradigm, where the conditions for publication are: are there any major errors, is there plagiarism, and does the work sufficiently provide evidence for the experimental hypothesis (note: I intentionally do not say "prove".). This system accepts the reality that we cannot sufficiently judge significance. Breakthroughs by definition come from research areas that are under studied. Frequently from directions that had previously been rejected. Unfortunately, this is becoming more common due to the speed in which researchers must publish and exacerbated by the rise in thresholds to get a work published. Many ideas are abandoned not due to evidence that they will not yield progress, but that the belief to achieve "sufficient" evidence would require too much time. While tenure should "resolve" this, it ignores the fact that a tenured professor still has students that must publish as well as they've been working in a very different way for years. And old habits are hard to break. I think we need to take a hard long look at our system and I suspect we could do so much more if we rebuilt the structure. [0] https://openreview.net/forum?id=cIKQp84vqN https://openreview.net/forum?id=cIKQp84vqN [1] https://scholar.google.com.hk/citations?user=D-TS1fAAAAAJ https://scholar.google.com.hk/citations?user=D-TS1fAAAAAJ
drawkward 2y ago
Happens in patents too. At one of my gigs, there was an engineering department that was perpetually being granted patents for their work. The C-level exec that the department laddered up to demanded on being on every patent app, despite having nothing to do with the invention itself. My boss insisted that we include other team members on our patent apps, even if they were not involved, as a way of improving morale. (It backfired. No one on the team who eas uninvolved wanted a patent they didnt earn. No one who earned the rigjt to be named on the patent app felt it was right to include non-inventors.)
tokai 2y ago
Interestingly King Fahd University, the affiliate of the two authors, has been accused of buying better metrics by getting successful academics to include the institution in their affiliations. https://english.elpais.com/science-tech/2024-12-05/dozens-of-the-worlds-most-cited-scientists-stop-falsely-claiming-to-work-in-saudi-arabia.html https://english.elpais.com/science-tech/2024-12-05/dozens-of...
dgfitz 2y ago
I wish the incentives would flip from “publish” to “reproduce” at this point. Or some 50/50 split. Obviously we need new studies. But the f they don’t replicate, what’s the point?
njarboe 2y ago
It might be good if the author list had a fraction attached to each author for their contribution share. It could be painful to negotiate this but would help with citation inflation.
silexia 2y ago
Hopefully Trump changes these incentive structures.
amai 2y ago
More people need to read the book https://en.wikipedia.org/wiki/Science_Fictions https://en.wikipedia.org/wiki/Science_Fictions Then this wouldn't be a surprise to anyone.

188 points by crescit_eundo 2y ago

https://archive.ph/SnYjB https://archive.ph/SnYjB

note that the 2% list is very large, thus most of the scientists are not just the leading edge superstars but also run of the mill scientists and principal investigators. The authors focus on the citation inflation in younger scientists but that s unfair (it's only 1000 out of the 20000 in the top 10%). The reality is that older established scientists are much more advantaged because the people of their lab are sought after as collaborators , and they autonatically get an authorship as last authors by virtue of being principal investigators, even if they don't even take a look at the paper. It's thus not strange that they get 35+ authorships per year

johnea 2y ago
This is pretty much what I came here to post. Anyone who's even been adjacent to scientific academia knows that the lead author is typically the advisor to the grad student or post doc that actually did the research and wrote the paper.
- asdff 2y ago
  In my experience the lead author is always the post doc or grad student who actually did the research and wrote the paper, while the advisor is the last author and they developed the grant to fund both the grad student and post doc. If you collaborate with another lab, their post docs go towards the front behind your name and the advisor goes somewhere at the end. If two people equally wrote a paper they might be 1 and 2 and get an asterisk indicating they are both co first authors. If anyone was involved at all in the work e.g data collection or anyone involved with running an assay they go somewhere in the middle. Perhaps it’s different in other fields
  - buildbot 2y ago
    Yep, this is what I learned in Grad school as well. First author is the primary person doing the work typically, then their advisor will be the last, and then you work towards the middle. So names in the middle of a paper typically contribute less than either edge!
  - jltsiren 2y ago
    This sounds like a terminological issue. Without further context, I would assume that "lead author" means the last/senior author, not the first/junior author. But I guess that's a life sciences tradition. "Who actually did the research" is not always an accurate description of the first author. There are plenty of papers, where the last author had already contributed enough to justify authorship before the first author was even hired. You often need to develop the idea and get preliminary results to convince someone to fund the project, before you get the money to hire the first author to work on the idea.
  - danielthor 2y ago
    When I was a grad student the only time my PI was listed first on a paper was when he actually did most of the work and I just helped with maths. Otherwise the PI was last and the grad students and post docs doing the research up front. People doing data collection or other ancillary work were mentioned in the acknowledgement section at the end. When the author count is much higher, like in the thousands in high energy physics, there are different norms like listing the authors alphabetically if the paper summarizes many significant achievements. In the in-between author order is negotiated.
- levocardia 2y ago
  ...and imagine how well it would go over for a grad student to say "actually Dr Advisor, you don't meet the criteria for authorship"
  - throwawaymaths 2y ago
    I've seen someone who desperately wanted their nobel laureate PI on their paper but the professor did not put his name on the paper because it was all their idea (even though it was his grant, he reviewed and ok'd the paper, paper was relevant to his lab, and he really spoke effusively at their thesis defense)
- zevon 2y ago
  This varies greatly by discipline and tradition - sometimes even within the same research groups. For example, some people put the senior person / advisor first, some put this person last, some go by alphabet. In my experience, there is even more variation in the amount of work expected to be put into a publication by said senior person. To put it mildly. ^^
mvdtnz 2y ago
These researchers compared the publish rates of Stanford Top 2% scientists with comparable Nobel laureates and the rate of publishing among the anomalous researchers was ludicrously higher than their peers at the top of their game - "up to 212 publications per year, and 5792 new coauthors per year, compared to up to 28.3 publications per year and 173 new coauthors per year on NL [nobel laureates]". Do you have any reason to believe the T2 group is composed of "older established scientists" who are "much more advantaged" than the group of Nobel Laureates? To the tune of 10x the number of publications and 33x coauthorships?
- seydor 2y ago
  > reason to believe the T2 group is composed of "older established scientists" there were 1000 academics with < 10 years of career and 19000 with more (in the inflators list). I did not compare their advantage to the nobel scientists but to the younger ones. (Also the nobel laureates are extreme outliers who don't need any more citations to satisfy their ego or funding needs)
dleeftink 2y ago
It's interesting how the observers of this preferential attachment effect have since become, well, very preferentially attached [0]. The scaling and diffusion of ideas is rarely (and unfortunately), equally distributed. [0]: https://scholar.google.com/scholar?q=Emergence+of+scaling+in+random+networks https://scholar.google.com/scholar?q=Emergence+of+scaling+in...
fph 2y ago
Authorship norms in certain fields seem completely broken: a PI shouldn't become an author just for providing money and a lab. Yet unfortunately that's the norm in many fields.
- fooker 2y ago
  > a PI shouldn't become an author just for providing money and a lab. What have the romans ever given us? https://www.youtube.com/watch?v=Qc7HmhrgTuQ https://www.youtube.com/watch?v=Qc7HmhrgTuQ
  - fph 2y ago
    I'm not claiming the money and lab are useless, but just don't call them "authors". When one publishes something with a grant from the NIH or the European Union or the Bill Gates foundation, they acknowledge the grant, they don't add the NIH or Bill Gates as an author. How is this different?

amelius 2y ago

The 10x scientists.

marginalia_nu 2y ago
To be fair, there has always been extreme outliers in scientific output, like Gauss and Euler.

LudwigNagasena 2y ago

> In our opinion, it is difficult to believe that the majority of authors with excessive rates are able to consistently produce large quantities of high quality or groundbreaking research, with their input to each paper being substantial and within norms of what constitutes co-authorship Huh, I thought providing the general direction and sanity-checking the results is the norm of what constitutes co-authorship for a top professor.

mvdtnz 2y ago
Do you think any one person can provide direction and sanity-check the work of more than 5,000 papers in a year? I don't.
- thinkingtoilet 2y ago
  Where did you get the 5000 number?
  - mvdtnz 2y ago
    From the research paper that the article is discussing. https://www.tandfonline.com/doi/full/10.1080/08989621.2024.2445280 https://www.tandfonline.com/doi/full/10.1080/08989621.2024.2... > Namely, rates on T2 of up to 212 publications per year, and 5792 new coauthors per year, compared to up to 28.3 publications per year and 173 new coauthors per year on NL. This is despite that the NL list is weighted with more senior researchers, often at the pinnacle of their careers – and hence – likely to have higher rates.
    - LPisGood 2y ago
      That is 212 papers, not 5000. 5000 is the number of new authors of those 200 papers.
      - Xelynega 2y ago
        Looks like the number is off, but is their point any different with 212 papers? I still think that it would be impossible to give any meaningful support to 212 papers in a year. That's a new paper every 1.72 days.

irrational 2y ago

So, who is the worst? Is there someone pumping out multiple new papers per day?

cgh 2y ago
From the article: "Similarly, paper mills – organizations that produce fabricated or low-quality manuscripts for profit – often rely on inflated co-authorship networks to boost the metrics of paying researchers..." The article references this paper on predatory journals as the source: https://link.springer.com/article/10.1186/s12916-015-0469-2?utm_source=getftr&utm_medium=getftr&utm_campaign=getftr_pilot&getft_integrator=tfo https://link.springer.com/article/10.1186/s12916-015-0469-2?... According to this paper more than 75% of the authors published in these journals originate in India and Africa (mostly Nigeria).

zzzeek 2y ago

[flagged]

UncleMeat 2y ago
Firing a shitload of people at random makes it more difficult to root out these things, not less.
rlili 2y ago
cool, will it do anything about CEOs which earn 100x what a common employee earns, despite not outputting 100x?
- kibwen 2y ago
  Happily, yes. They will now earn 10,000x what a common employee earns while doing less than ever before.
- BeetleB 2y ago
  In many (traditional) companies, a CEO's mistake can easily cause 100x more harm than an average worker's mistake.
  - nosianu 2y ago
    And when that happens, what happens to them? There "risk" is to get a lot of "golden parachute" money to leave earlier than their contract says, as a maximum penalty. If there us actual money to pay the company or insurance does it for them. . The human network of processes, laws, culture, knowledge established over millennia, which enables those CEOs to be at the top of such valuable companies in the first place was built by billions over time. Those CEOs are not bette than others, or people of the past, they are the same humans. That they now move billions instead of having a small stone axe business for the local tribes is not of their making. They were lucky. If you want to see someone really earning the hundred million with their own hard work, place them in the middle of nowhere, and with no access to the global human network that they did nothing whatsoever to construct because it was already there. The value that CEOs and others grab for themselves is not of their making. They are able to have such huge businesses only because a human network in space (the current one) and time (handed forward over time, knowledge and processes and tools). They "deserve" the millions just as much as everybody else. They misuse their superior position in the network, since they sit in places where lots of value not of their own making is going through. In addition to that, in many important cases they managed to catch all the upside, but let all the failures fall through to be caught be everybody else. I have nothing whatsoever against true entrepreneurs. I know a few personally! But as you reach the billions, and the top management (instead of direct owners who actually suffer when the company fails to deliver), the justification for the incomes gets more tenuous IMO. That is one of my big issues with the "Left" - here in Germany their new program includes getting rid of billionaires. It's too simple and too one-dimensional. The diversity of people in such high-value positions is huge, and there are many more dimensions! The left treat it like a one-dimensional scale, starting with a normal person that has a few dollars or Euros, and on that scale and idea of "money" they remain with their arguments. Instead, we have to look at the many many different things all that money actually stands for. CEO income with their usually much reduced risk compared to owners should not be treated the same, for example and IMHO.

readyplayernull 2y ago

Among given population X% are suspicious and Y% keep an eye on statistic and their heads below the radar.

recursivedoubts 2y ago

Academia is filled to the absolute brim with scams, news at 11. (I'm in academia)

gjsman-1000 2y ago
Yet they still demand we take them with utmost seriousness...
- pazimzadeh 2y ago
  not really, maybe you're thinking of medical doctors
matthewdgreen 2y ago
All human activity is full of scams if you cast your net wide enough. Insofar as academia contains scams, the distinction is that they have a much lower profit margin.
- cwmoore 2y ago
  How does one compute the profit margin of academia?
  - matthewdgreen 2y ago
    You look at the bad actors (in this case and others) and see how much money or equivalent they're getting out of their work. My general view is that it's much less than you could obtain with a PhD and similar effort applied to other fields.
- cherryteastain 2y ago
  > Sayre's law states, in a formulation quoted by Charles Philip Issawi: "In any dispute the intensity of feeling is inversely proportional to the value of the issues at stake." By way of corollary, it adds: "That is why academic politics are so bitter." https://en.wikipedia.org/wiki/Sayre%27s_law https://en.wikipedia.org/wiki/Sayre%27s_law

TrackerFF 2y ago

If you scrutinize the top publishers, I'm sure you'll find some duplicate publications, salami slicing, and other things. When you're that prolific, the number of publications also become a goal, especially if you're awarded for it. Maybe not outright academic fraud, but at least publishing pattern that tries to maximize some measure(s).

YossarianFrPrez 2y ago

I was just thinking about the upper bound on how many papers one could publish in a year earlier this morning. Speaking as someone who spent years in Tech and is now in the middle of a Ph.D. program... Within the current paradigm, where a post-doc gets hired as a new professor and goes about starting the rough equivalent of a single private sector team, at least in my subfield, 15-25 (non-first author publications) a year is an impressive number. And thus the numbers cited in the supplemental materials, the max being 136 papers a year, is strange, and I am pretty sure the author's points about paper mills etc. hold true. This is the great thing about the "Contributor Roles Taxonomy" system: it provides a lower level of abstraction and gives credit for who did what (idea generation, coding, writing, reviewing, raising the money, etc.) compared with using "a publication" at the unit of measure. It really solves a lot of problems. [1] [1] https://authorservices.wiley.com/author-resources/Journal-Authors/open-access/credit.html https://authorservices.wiley.com/author-resources/Journal-Au... But I'd also like to raise another point. People who wind up in Academia tend to go straight from undergrad to grad school (or spend a year being a lab manager in academia) and so most if not all of the systems we use in software development aren't present. Code review, project timeline estimation, building up a lab-wide codebase of functions to speed up repetitive tasks, 360 degree reviews, lab-wide project management software, an org-chart deeper than two (or in rare cases, three) layers, teams with differentiated responsibilities multiple teams, etc. are not the norm. Every so often I hear of one lab here or there that does one or two of these, not all of them. (Though my experience is limited.) My point is that if one were to apply all of the modern systems used for coordinating groups of people to produce structured forms of writing, etc., then 100 papers a year sans a breadth-vs.-depth tradeoff might just be doable. But note that this is not "100 papers as year" from an individual, it's "100 papers a year from a mid-sized institution." (Six teams of four getting out 1.5 papers a month equates to 108 papers a year, near the maximum cited above.) Bell Labs' publication / patent rate must have been high! Granted, what I'm saying is not exactly within the current paradigm of how science is done, and might not be possible in a university setting.

Steuard 2y ago

It's quite possible that there's a real effect here. But while I've only had time to skim parts of the paper, I don't see any indication of whether the authors have accounted for the different norms in different fields when analyzing their data for potentially fraudulent or deceptive behavior. Just for example, physics papers produced by large international collaborations (e.g. every single paper from the Large Hadron Collider at CERN) routinely have hundreds of authors (e.g. https://www.nature.com/articles/nature.2015.17567 https://www.nature.com/articles/nature.2015.17567): everyone who has made substantial contributions to the design and operation of the facility is listed, as is everyone on the data analysis teams. (My understanding is that people in those specific fields all recognize that "number of citations" is a mostly meaningless number for those involved, and other metrics for productivity are well-known in those communities and routinely used.) I hear that some genomics papers have broken 1000 authors as well. I could easily imagine that the high end of observed publication numbers and coauthor counts would be dominated by those giant collaborations, even though there is absolutely no attempt to mislead anyone in the process. Can anyone tell from this article to what degree its conclusions might be influenced by this factor?

daft_pink 2y ago

We’re going to penalize people for publishing too mucn now? Really? Just want to say that I love Issac Asimov and he wrote a lot of high quality work and people even say that he’s a graphomaniac with a writing compulsion. I think people’s work should be judged by the output quality not some sort of speed limiting judgment.

kergonath 2y ago
> Just want to say that I love Issac Asimov and he wrote a lot of high quality work and people even say that he’s a graphomaniac with a writing compulsion. There is a difference between being prolific and what those people put out. At least one order of magnitude. Isaac Asimov did not write 80 novels each year.
Xelynega 2y ago
Isaac Asimov wroted/edited ~500 books over his life. Some of the people mentioned in the research are coauthoring ~200 paper a year. I don't think Isaac would be in the same league as those mentioned in this paper.
robertlagrant 2y ago
Isaac Asimov's work was judged by how many people published it, and how many people bought it. There's no equivalent quality control for these papers; it's much more the honour system.
staticman2 2y ago
This is more like criticizing Asimov for allowing a magazine to be published called "Isaac Asimov's Science Fiction Magazine" where he was not the editor and it was not his magazine.

userbinator 2y ago

I wonder how many of these papers are now mostly if not completely AI-generated. This happened a year ago, and only gained attention because it was bad enough to be noticed: https://news.ycombinator.com/item?id=39391034 https://news.ycombinator.com/item?id=39391034

alberth 2y ago

Isn't this a well-known issue in the academic and research community where researchers include their friends' names on their published work? Since researchers are rated and measured by the number of published papers they have, many people game the system by exchanging bylines with their friends ... so that it boosts their total number of papers published.

mrweasel 2y ago
This was one of the ethical dilemmas we had to go through at university some 20 years ago: "Your professor/supervisor tells you to add them to the list of authors on your paper, do you do it?". There's a whole subset of "issues" here, one where you're more likely to be published with the supervisors name on the paper, another where you're travelling to present your paper, but you're not likely to be granted the budget and adding a professors name will almost ensure an all expenses paid trip. Low hundreds papers doesn't seem impossible, nor does hundreds of collaborators, but it would heavily depend on your work. We see the same in software. Some people seems unrealistically productive, spawning one successful project after another. I would be interested in knowing for how long they can keep up the pace though. There's also the question of the quality of their work.

thinkingQueen 2y ago

My eyes were opened when a field called gamification appeared in the early 2010s. In a few years many gamification researchers had tens of thousands of citations, h-indexes nearing hundred. Well, if you think about it they’re gamers, they’ve been grinding their RPG characters, sniping skills and whatnot for thousands of hours. It’s only natural that these guys and girls figure out how to reach the maximum scientific high score. Some of the gamification researchers are near the top 500 of that 2% list. Now ask yourself, is gamification something that should make you one of the top 500 scientist in the world? I doubt it, but modern science is a citation game. Nothing else.

hansvm 2y ago
That feels like a certain xkcd [0]. If you can game the totally bullshit rankings then clearly your papers are worth something. [0] https://xkcd.com/2385/ https://xkcd.com/2385/
- lmm 2y ago
  Thought that was going to be https://xkcd.com/125/ https://xkcd.com/125/. Iteration of the same concept I guess.
xdavidliu 2y ago
reminds me of the fraudulent Ariely and Gino papers that were exposed a year or two ago, where a very common comment was "they're dishonesty researchers, of course they are going to be dishonest".
layman51 2y ago
Gamification as a field of study also reminds me a lot of this video[1] of a talk about tips for game developers. It is very interesting because it is from 2016 which is close to the time period you mentioned. I’m pretty sure this video has been shared on Hacker News a bunch. [1]: https://www.youtube.com/watch?v=xNjI03CGkb4 https://www.youtube.com/watch?v=xNjI03CGkb4

godelski 2y ago

It amazes me how many "high profile" researchers have blatantly fraudulent work. As an example, take this rather recent ICLR submission that is egregiously plagiarized[0] (I mean attempts to hide the plagiarism are minimal to nonexistent). Clicking on the authors you can view their profiles and you'll find that most authors have several thousand citations and one north of 25k[1]!!!! They also have an i-10 of 400, which is highly abnormal for that citation level while the last author has an i-10 of 126 with only 7.3k citations. The problem I see here is that there's lots of suspicious activity and clear evidence of fraud, yet little to no action is taken. If anything, it seems we mostly ignore it. The ICLR submission is a bit odd too, since decisions and comments are public, most of the time this happens behind closed doors and we don't observe the evidence. As far as I'm aware, no one is making network graphs to track fraudulent authors. I think we want to sweep it under the rug because at the end of the day our system relies heavily upon trust. Unless we strongly incentivize reproduction we require high trust and we're afraid to acknowledge fraud in the community because we believe it will make more distrust science. But I think in reality, the reluctance to pursue fraud and the reluctance to have high clarity creates more distrust. The truth is that this system poorly scales even before we incorporate the reverse (bureaucratic) incentives. I think we lost sight of the goals of research and publishing. Publishing exists to facilitate communication among our peers. We publish in venues for two reasons: to improve/error check works and to facilitate social networks. Publish or perish is detrimental as well as the push for highly subjective measures like novelty. I strongly advocate for returning to the old paradigm, where the conditions for publication are: are there any major errors, is there plagiarism, and does the work sufficiently provide evidence for the experimental hypothesis (note: I intentionally do not say "prove".). This system accepts the reality that we cannot sufficiently judge significance. Breakthroughs by definition come from research areas that are under studied. Frequently from directions that had previously been rejected. Unfortunately, this is becoming more common due to the speed in which researchers must publish and exacerbated by the rise in thresholds to get a work published. Many ideas are abandoned not due to evidence that they will not yield progress, but that the belief to achieve "sufficient" evidence would require too much time. While tenure should "resolve" this, it ignores the fact that a tenured professor still has students that must publish as well as they've been working in a very different way for years. And old habits are hard to break. I think we need to take a hard long look at our system and I suspect we could do so much more if we rebuilt the structure. [0] https://openreview.net/forum?id=cIKQp84vqN https://openreview.net/forum?id=cIKQp84vqN [1] https://scholar.google.com.hk/citations?user=D-TS1fAAAAAJ https://scholar.google.com.hk/citations?user=D-TS1fAAAAAJ

drawkward 2y ago

Happens in patents too. At one of my gigs, there was an engineering department that was perpetually being granted patents for their work. The C-level exec that the department laddered up to demanded on being on every patent app, despite having nothing to do with the invention itself. My boss insisted that we include other team members on our patent apps, even if they were not involved, as a way of improving morale. (It backfired. No one on the team who eas uninvolved wanted a patent they didnt earn. No one who earned the rigjt to be named on the patent app felt it was right to include non-inventors.)

tokai 2y ago

Interestingly King Fahd University, the affiliate of the two authors, has been accused of buying better metrics by getting successful academics to include the institution in their affiliations. https://english.elpais.com/science-tech/2024-12-05/dozens-of-the-worlds-most-cited-scientists-stop-falsely-claiming-to-work-in-saudi-arabia.html https://english.elpais.com/science-tech/2024-12-05/dozens-of...

dgfitz 2y ago

I wish the incentives would flip from “publish” to “reproduce” at this point. Or some 50/50 split. Obviously we need new studies. But the f they don’t replicate, what’s the point?

njarboe 2y ago

It might be good if the author list had a fraction attached to each author for their contribution share. It could be painful to negotiate this but would help with citation inflation.

silexia 2y ago

Hopefully Trump changes these incentive structures.

amai 2y ago

More people need to read the book https://en.wikipedia.org/wiki/Science_Fictions https://en.wikipedia.org/wiki/Science_Fictions Then this wouldn't be a surprise to anyone.