5 ms·

This may be naive but why not use something like PageRank as an objective measure of a paper (and the author's) quality? Of course this can be computed only af

This may be naive but why not use something like PageRank as an objective measure of a paper (and the author's) quality? Of course this can be computed only after the paper is already published...

chrisseaton 9y ago
You mean count citations? Is being cited a lot the same thing as being high quality? What about a paper that is continually refuted by other papers? Is that a high quality paper?
- nilkn 9y ago
  Perhaps the word should be "notable" rather than "high-quality." A paper that is repeatedly refuted must have had some intellectual impact if others have bothered to refute it in so many ways. PageRank would also take into account the quality or notableness of the refuting papers themselves, so this cannot be reduced to a mere count of the number of refutations. That means a highly-ranked, oft-refuted paper would have been refuted not just by anyone, but by other high-quality or at least highly notable authors.
- dhimes 9y ago
  I believe that system existed before pagerank, and always assumed that Larry and Sergey were inspired by it. EDIT: CiteSeer is what I was thinking of (1997).
mayank 9y ago
Fun fact: the eigenvector centrality computation in PageRank was originally used for citation networks many years before PageRank. A surprising number of web spam problems have parallels in academic publishing: spam rings, "rich get richer" problems caused by paper search engines, and outright demands for self-citations from editors to boost a journal's impact factor.
asdfgadsfgasfdg 9y ago
Like impact factor? This is done already and it is very flawed.
carbocation 9y ago
I agree, which is why I'm building https://pubrank.carbocation.com https://pubrank.carbocation.com to do just that. One of the more challenging things about PubMed and its 27 million indexed papers is that there is virtually no sense of a unique author identifier, which has made that aspect somewhat of a research project for me. (It's an active area of research on which a few CS papers have been published. I'm trying to hack through it with some older statistical methods for a "good enough" solution.)
neuromantik8086 9y ago
It's been done: http://www.eigenfactor.org/ http://www.eigenfactor.org/ I don't think tenure committees pay attention to it because it's not as widely accepted/established. Individuals still on the tenure-track career path are welcome to correct me if I'm wrong though.
- neuromantik8086 9y ago
  Nvm. Eigenfactor is at the level of journals, not individuals- my brain doesn't work today. Someone did try to calculate some centrality measures for researchers in neuroimaging though: http://cos.dery.xyz/ http://cos.dery.xyz/ This would also probably not be taken seriously by a TT committee.
afpx 9y ago
For the past decade or so, I've often wondered why, given the informational tools available, research is condensed into 'a paper'. I hate reading 'papers' because there's so much left out. I want to know about the initial hypothesis, the drivers, the paths taken, the false starts and changes in direction, and of course the failures. There's so much value lost. The 'paper' seems the scientific equivalent of the photoshop edit.

This may be naive but why not use something like PageRank as an objective measure of a paper (and the author's) quality? Of course this can be computed only after the paper is already published...

chrisseaton 9y ago

You mean count citations? Is being cited a lot the same thing as being high quality? What about a paper that is continually refuted by other papers? Is that a high quality paper?

nilkn 9y ago
Perhaps the word should be "notable" rather than "high-quality." A paper that is repeatedly refuted must have had some intellectual impact if others have bothered to refute it in so many ways. PageRank would also take into account the quality or notableness of the refuting papers themselves, so this cannot be reduced to a mere count of the number of refutations. That means a highly-ranked, oft-refuted paper would have been refuted not just by anyone, but by other high-quality or at least highly notable authors.
dhimes 9y ago
I believe that system existed before pagerank, and always assumed that Larry and Sergey were inspired by it. EDIT: CiteSeer is what I was thinking of (1997).

mayank 9y ago

Fun fact: the eigenvector centrality computation in PageRank was originally used for citation networks many years before PageRank. A surprising number of web spam problems have parallels in academic publishing: spam rings, "rich get richer" problems caused by paper search engines, and outright demands for self-citations from editors to boost a journal's impact factor.

asdfgadsfgasfdg 9y ago

Like impact factor? This is done already and it is very flawed.

carbocation 9y ago

I agree, which is why I'm building https://pubrank.carbocation.com https://pubrank.carbocation.com to do just that. One of the more challenging things about PubMed and its 27 million indexed papers is that there is virtually no sense of a unique author identifier, which has made that aspect somewhat of a research project for me. (It's an active area of research on which a few CS papers have been published. I'm trying to hack through it with some older statistical methods for a "good enough" solution.)

neuromantik8086 9y ago

It's been done: http://www.eigenfactor.org/ http://www.eigenfactor.org/ I don't think tenure committees pay attention to it because it's not as widely accepted/established. Individuals still on the tenure-track career path are welcome to correct me if I'm wrong though.

neuromantik8086 9y ago
Nvm. Eigenfactor is at the level of journals, not individuals- my brain doesn't work today. Someone did try to calculate some centrality measures for researchers in neuroimaging though: http://cos.dery.xyz/ http://cos.dery.xyz/ This would also probably not be taken seriously by a TT committee.

afpx 9y ago

For the past decade or so, I've often wondered why, given the informational tools available, research is condensed into 'a paper'. I hate reading 'papers' because there's so much left out. I want to know about the initial hypothesis, the drivers, the paths taken, the false starts and changes in direction, and of course the failures. There's so much value lost. The 'paper' seems the scientific equivalent of the photoshop edit.