2 ms·
One Scientist's Crusade to Rewrite Reputation Rules
- thras 17y agoUnfortunately, both systems are too easy to game by those who happen to be more unprofessional than average. In the end, these sorts of systems are foisted on us by the paid bureaucrat-class that pays itself quite well for doing all that really hard work of managing academics. Figuring out whether someone is a hotshot scientist would mean reading his papers, and that's way too much work.
- knowtheory 17y ago1) Whether someone is a hotshot scientist is a subjective matter of opinion. 2) The question they are asking is not "How important are your contributions to science?", but "How important do your peers think your contributions to science are?" I'd probably go as far as to claim that it's nonsensical to search for an objective value metric for contributions to science. Scientific contributions are extremely heterogeneous, and value judgements are equally varied.
- amichail 17y agopeers = friends?
- thras 17y agoYes. There is no pure objective measurement, but...You still need to hire some people and not hire others. Value judgments are not randomly distributed. Science is a built upon forging objective consensus out of individual opinion. It even works pretty well.
- knowtheory 17y agoI'm not against ranking people, although i do think we need to keep in mind what the rankings really mean. And, i'd say that the rankings work well enough, but they're certainly not perfect, as pointed out by the article.
- Herring 17y agoThere probably isn't an objective value metric for contributions to the internet either.
- sachmanb 17y agothis is almost pagerank for academic publications, and would probably be improved by being more like pagerank (where it also took into account how often citing papers were cited)
- amichail 17y agoThe very people who complain about having their research being ignored in prestigious venues would probably not do well with a PageRank-like measure.
- sachmanb 17y agoregardless of whether it is the right thing to do, what he is trying to do could be improved by being more like pagerank. i don't have an opinion on whether it is the right thing to do; i haven't given that much thought.
- Herring 17y agoPresumably they'd do better than with the JIF system. Orthodoxy isn't the problem. He's still getting published & cited. He just can't get grants apparently because of his 1989 conference.
- timr 17y agoThere are a lot of people trying to use pagerank for academic journals, but so far it hasn't worked well for various reasons. Part of the problem is that the metaphor breaks down: a paper is like an individual webpage, but a journal is like a company -- it has a much longer time-line, and its impact varies over time. Also, unlike web links, citations don't go away; they just accumulate over time. Since the point of these citation metrics are to rate the journals (and maybe the scientists), pagerank has some difficulties in the domain. It works better for ranking individual papers than for scientists or their journals. This shouldn't be too surprising: TechCrunch (for example) probably has a good rank on many pages, but pagerank doesn't tell us anything about Michael Arrington's reputation.
- chancho 17y ago
- lutorm 17y agoStill has all the problems that counting citations do: how do you count multiple-author papers (particle physics has given up ranking authors and just put the hundreds of authors in alphabetical order, IIRC) and it encourages people to write piecemeal papers, because why say something in one paper if you can say it in two -- and get twice the citations? Also, my favorite: What if you cite someone to say that they were completely wrong when they approached the same problem? Should that be counted as a positive for them?
- natrius 17y agoPresumably, no one is going to try to prove something wrong that has already been proven wrong, so unless there are many obscure flaws in one's work that take multiple papers to uncover, it shouldn't affect the citation count much.
- lutorm 17y agoIn introductions, papers often give a short overview of what's been done before, so early work in a field often gets cited even if it turns out to have been flawed. (It can't be completely ridiculous, of course, it must have sounded plausible at the time, but even shoddy research often does.)