4 ms·

Imo, it has to do with incentives and rewards.(in CS at the very least) The Hindex is a measure thats become the target. So 1 seminal paper is much much worse

by screye 5y ago

Imo, it has to do with incentives and rewards.(in CS at the very least) The Hindex is a measure thats become the target. So 1 seminal paper is much much worse than 10 completely forgettable papers. It is also common to divide seminal work into smaller publishable morsels to rack up the hindex at the cost of novelty and clarity. The lack of incentive to pursue novel works, also means that most papers are incremental by design. Imo, this is the biggest waste of money at academic phd programs. However, the students need to take their career somewhere, so an uncomfortable compromise is met. Another problem is how every conference is converging into the same impact maximizing mush without any meaningful differences between them. This has massively affected searchability, which as we know leads to even greater 'agglomeration at the top'. Having different standards for novelty, experimental rigor, math rigor, scale, technical fit and the like would allow for accepted papers to follow internally consistent searchable constraints, while the diversity of target audiences accross conferences would allow for different types of research to coexist. In my field, conferences are only differentiated by deadlines and status. Everything else is secondary.There arent too many papers. There just that the quality of curation has gone to shit.

carbocation 5y ago
It's painful because (IMHO) the H index is just a much worse approximation of something that we could actually achieve with PageRank for academic citations. In that case, a bunch of middling papers would be rewarded, but so too would one critical paper that lays a foundation for a field.
- prionassembly 5y ago
  Wouldn't betwenness centrality be better? Think of an army randomly moving through your citation graph; the more particularly nodes are trampled over, the more pagerank it has. Now: if this army is informed about the shortest routes and instead moves about optimally, the most-trampled over places have higher betweenness centrality. I'd like my simulated citing scientist to be smart.
  - carbocation 5y ago
    I've tried applying PageRank to pubmed with reasonable results, but betweenness does sound compelling. Thanks!
    - prionassembly 5y ago
      I'll give you a bonus: the Louvain algorithm for community detection. Whatever ships with Networkx, Gephi, etc. doesn't work for my (correlation-derived, pruned with graphical lasso) networks, but the Louvain method (a greedy approximation to modularity maximization; the real math magic is in the concept of modularity and the configuration model) is awesome.
      - carbocation 5y ago
        Interesting. We use Louvain for single nucleus sequencing cell type identification. Cool to think about its other applications. Thanks for sharing.
- lrem 5y ago
  I always thought that PageRank was inspired on methods previously used for scoring academic papers. Now I'm wondering if I misunderstood something, or my professor misunderstood it first. Damn.
  - generationP 5y ago
    PageRank has nothing to do with h-index to my knowledge. The problem with using PageRank-like methods for academic papers is that academic papers mostly reference backwards in time (except for the occasional draft or work-in-progress being referenced, but proper cycles are rare). A triangular matrix doesn't yield an interesting stationary distribution...
    - fooofw 5y ago
      I don't know if it makes any difference, but if PageRank should replace the h-index, the most direct equivalent would be to rank the authors rather than the papers, no?
    - andyferris 5y ago
      You could just make it bidirectional (add the transpose and make it symmetric), right?
      - generationP 5y ago
        Too easy to game by citing the most reputed sources. Reputation shouldn't flow up a citation.
luckluckgoosed 5y ago
Doesn't h-index prevent exactly the scenario of having 10 forgettable papers? Having a couple of great papers yields a high h-index, but 10 forgettable papers would hold the number at a low count because you need N papers with at least N citations each. So a seminar paper would +1 to that h-index indefinitely, whereas low value papers would upper bound the h-index to their citation count.
- screye 5y ago
  It kind of regresses to a median. 1 big paper, and 100 papers with zero citations aren't that useful. However, most top phd students/ assistant professors hover around the nebulous 5-30 hindex where getting 30 citations is a lot easier than publishing 30 papers. So, in most cases, you will prefer to figure out quantity, because the quality bar is so low. Additionally, they and lab mates always cite each other which leads to a free 10-ish citations overtime anyway. Lastly, authorship priority is not taken into account in hindex. So, a bunch of secondary-authors can easily get those numbers up at massively industrialized labs. So a small set of productive 1st authorships are given lower weightage than a large list of low-contribution 2nd authorships. Almost all super-high hindex professors are more like CEOs of a research company than primary researchers. H-index, like all metrics is useful. It sort of shows the median quality of papers by an author assuming that equal time is spent on all papers. It is informative, but making it too important in academia has led to it getting gamed with counter-productive incentive structures. H-index ignores away a lot qualities that are incredibly important to being a productive researcher, and has led to researchers with such qualities being progressively pushed out of academia ever since it has become THE target.
  - ta988 5y ago
    > assuming that equal time is spent on all papers. And that's exactly what makes the h-index useless, every paper has a different effort put into it. I've had papers that required 5y of work, some others only 1 month... And yet they are counted the same. Also the number of citations depends on the field, for people working across multiple fields you get papers that are highly important for a smaller community get less citations than low quality papers in a large community... Most papers are cited "by chance". Researcher type two keywords, cite paper that seem to go somewhat toward the wanted direction, rinse repeat.
chinchilla2020 5y ago
The incentives in academia are to publish and receive citations. This is often very political and citations are made as a favor or trade, completely independent of the research. Unfortunately this structure forces out productive scientists who want to do interesting research rather than churn papers.
- deleted 5y ago
  [deleted]
hardtke 5y ago
You accidentally conflated "conference" and "paper" which is true in computer science and also relevant to other fields. The crux of the problem is that almost no one reads papers. People get exposed to ideas at conferences, and conference organizers invite the people that wrote the seminal papers many years ago to be the keynote speakers. Even if a paper is cited, it is usually because they saw the author speaking at a conference.
jltsiren 5y ago
The bigger problem is structural. Academic research is mostly done by early-career researchers in short-term positions. When you are a PhD student or a postdoc with only a few years to do research, you don't have the time for long-term projects. And because your career prospects are determined by your success in your current position, you have to manage the risks by working on multiple projects. By the time they graduate, the average successful PhD will then have several publishable results where they were the primary contributor. There are very few positions for mid-career individual contributors in the academia. After ~10 years, you are expected to take a leadership position. Instead of doing research yourself, you are now expected to convince your peers that your ideas are worth funding and that they should collaborate with you. If you get the funding, you will hire people to do your research. Your job is then guiding them through the early career. You can start thinking about ambitious long-term projects, but in order to make them viable, you have to break them down into smaller publishable subprojects suitable for your students.

by screye 5y ago

carbocation 5y ago

It's painful because (IMHO) the H index is just a much worse approximation of something that we could actually achieve with PageRank for academic citations. In that case, a bunch of middling papers would be rewarded, but so too would one critical paper that lays a foundation for a field.

prionassembly 5y ago
Wouldn't betwenness centrality be better? Think of an army randomly moving through your citation graph; the more particularly nodes are trampled over, the more pagerank it has. Now: if this army is informed about the shortest routes and instead moves about optimally, the most-trampled over places have higher betweenness centrality. I'd like my simulated citing scientist to be smart.
- carbocation 5y ago
  I've tried applying PageRank to pubmed with reasonable results, but betweenness does sound compelling. Thanks!
  - prionassembly 5y ago
    I'll give you a bonus: the Louvain algorithm for community detection. Whatever ships with Networkx, Gephi, etc. doesn't work for my (correlation-derived, pruned with graphical lasso) networks, but the Louvain method (a greedy approximation to modularity maximization; the real math magic is in the concept of modularity and the configuration model) is awesome.
    - carbocation 5y ago
      Interesting. We use Louvain for single nucleus sequencing cell type identification. Cool to think about its other applications. Thanks for sharing.
lrem 5y ago
I always thought that PageRank was inspired on methods previously used for scoring academic papers. Now I'm wondering if I misunderstood something, or my professor misunderstood it first. Damn.
- generationP 5y ago
  PageRank has nothing to do with h-index to my knowledge. The problem with using PageRank-like methods for academic papers is that academic papers mostly reference backwards in time (except for the occasional draft or work-in-progress being referenced, but proper cycles are rare). A triangular matrix doesn't yield an interesting stationary distribution...
  - fooofw 5y ago
    I don't know if it makes any difference, but if PageRank should replace the h-index, the most direct equivalent would be to rank the authors rather than the papers, no?
  - andyferris 5y ago
    You could just make it bidirectional (add the transpose and make it symmetric), right?
    - generationP 5y ago
      Too easy to game by citing the most reputed sources. Reputation shouldn't flow up a citation.

luckluckgoosed 5y ago

Doesn't h-index prevent exactly the scenario of having 10 forgettable papers? Having a couple of great papers yields a high h-index, but 10 forgettable papers would hold the number at a low count because you need N papers with at least N citations each. So a seminar paper would +1 to that h-index indefinitely, whereas low value papers would upper bound the h-index to their citation count.

screye 5y ago
It kind of regresses to a median. 1 big paper, and 100 papers with zero citations aren't that useful. However, most top phd students/ assistant professors hover around the nebulous 5-30 hindex where getting 30 citations is a lot easier than publishing 30 papers. So, in most cases, you will prefer to figure out quantity, because the quality bar is so low. Additionally, they and lab mates always cite each other which leads to a free 10-ish citations overtime anyway. Lastly, authorship priority is not taken into account in hindex. So, a bunch of secondary-authors can easily get those numbers up at massively industrialized labs. So a small set of productive 1st authorships are given lower weightage than a large list of low-contribution 2nd authorships. Almost all super-high hindex professors are more like CEOs of a research company than primary researchers. H-index, like all metrics is useful. It sort of shows the median quality of papers by an author assuming that equal time is spent on all papers. It is informative, but making it too important in academia has led to it getting gamed with counter-productive incentive structures. H-index ignores away a lot qualities that are incredibly important to being a productive researcher, and has led to researchers with such qualities being progressively pushed out of academia ever since it has become THE target.
- ta988 5y ago
  > assuming that equal time is spent on all papers. And that's exactly what makes the h-index useless, every paper has a different effort put into it. I've had papers that required 5y of work, some others only 1 month... And yet they are counted the same. Also the number of citations depends on the field, for people working across multiple fields you get papers that are highly important for a smaller community get less citations than low quality papers in a large community... Most papers are cited "by chance". Researcher type two keywords, cite paper that seem to go somewhat toward the wanted direction, rinse repeat.

chinchilla2020 5y ago

The incentives in academia are to publish and receive citations. This is often very political and citations are made as a favor or trade, completely independent of the research. Unfortunately this structure forces out productive scientists who want to do interesting research rather than churn papers.

deleted 5y ago
[deleted]

hardtke 5y ago

You accidentally conflated "conference" and "paper" which is true in computer science and also relevant to other fields. The crux of the problem is that almost no one reads papers. People get exposed to ideas at conferences, and conference organizers invite the people that wrote the seminal papers many years ago to be the keynote speakers. Even if a paper is cited, it is usually because they saw the author speaking at a conference.

jltsiren 5y ago

The bigger problem is structural. Academic research is mostly done by early-career researchers in short-term positions. When you are a PhD student or a postdoc with only a few years to do research, you don't have the time for long-term projects. And because your career prospects are determined by your success in your current position, you have to manage the risks by working on multiple projects. By the time they graduate, the average successful PhD will then have several publishable results where they were the primary contributor. There are very few positions for mid-career individual contributors in the academia. After ~10 years, you are expected to take a leadership position. Instead of doing research yourself, you are now expected to convince your peers that your ideas are worth funding and that they should collaborate with you. If you get the funding, you will hire people to do your research. Your job is then guiding them through the early career. You can start thinking about ambitious long-term projects, but in order to make them viable, you have to break them down into smaller publishable subprojects suitable for your students.