3 ms·

Rankings like ARWU or US News are quite biased towards big research universities. Nature and others have released rankings that are a bit normalized by size, a

by nextos 5y ago

Rankings like ARWU or US News are quite biased towards big research universities. Nature and others have released rankings that are a bit normalized by size, and many niche institutions such as Cold Spring Harbor or the Austrian IST are near the top. This was discussed in an old HN post, which is quite interesting to read through: https://news.ycombinator.com/item?id=20231432 https://news.ycombinator.com/item?id=20231432. ARWU subject ranking is pretty good, though. Rankings that are not subject based are hard to interpret. For example, many smaller Ivies have excellent departments in topic X but are not so good in topic Y. They can't cover everything. But if you trust the ranking and go there anyway to study topic Y, you might be disappointed. However, funding is really biased towards big or famous institutions. In my experience, it's really easy to get tons of grants from a place like Oxford or Cambridge. If you move to an excellent place that is less known, say a university in the top 30-50 of the world, it gets a lot harder even if your proposal is identical.

PeterisP 5y ago
There's also a big implicit bias towards the concept that research is done in universities - in some places there's a distinction of institutions where universities focus on teaching and most of academic research is done outside of universities in separate research institutes (e.g. the Fraunhofer institute chain in Germany which employs many thousands of researchers and has resulted in a lot of high-impact science); a professor might be involved in both, but his research work wouldn't count for the university's rating (because it's not affiliated with the university) and the institutes are excluded from that rating because they don't teach undergrads.
- nextos 5y ago
  That's true. Some rankings like the one from Nature I mentioned do include institutes. As a result, places like Wellcome Sanger in Cambridge or Helmholtz in Germany rank really well and above most leading universities in their field. As you say, these are not universities, but research institutes and therefore ignored by ARWU, US News and others.
- IanCal 5y ago
  Rankings can also be affected depending on how the identification of the university in the affiliation is done. Automated matches are often lower for languages that have different ways of writing things (often when there are different characters, or it's written in different languages), changes to location/name/structure, and how long it's been around. Based on a quick look at wikipedia: For Szeged, the university is kind of old but also only appeared in it's current form in 2000. It's also called Szegedi Tudományegyetem. It has multiple colleges inside it, so it may depend on how well those are represented in the data. Authors using acronyms more makes identification less likely, each of the colleges have acronyms. Finally, where authors publish makes a big difference - not all publishers open their data up and even those that allow access to some raw data it can be less reliable to extract. So if you publish in places where the publisher does not make the affiliations easily machine readable your work may be counted less in many of these automated rankings. Pubmed (not a publisher but a key way a lot of people get some of this data) didn't have affiliations for anyone but the first author until some fairly late date. Either automated mappings are used for a lot of work (since there are tens of millions of unique affiliation strings), or the source material is heavily curated (so for example only the "top" N journals are looked at). source: worked on affiliation mapping at Dimensions in a previous life

by nextos 5y ago

PeterisP 5y ago

There's also a big implicit bias towards the concept that research is done in universities - in some places there's a distinction of institutions where universities focus on teaching and most of academic research is done outside of universities in separate research institutes (e.g. the Fraunhofer institute chain in Germany which employs many thousands of researchers and has resulted in a lot of high-impact science); a professor might be involved in both, but his research work wouldn't count for the university's rating (because it's not affiliated with the university) and the institutes are excluded from that rating because they don't teach undergrads.

nextos 5y ago
That's true. Some rankings like the one from Nature I mentioned do include institutes. As a result, places like Wellcome Sanger in Cambridge or Helmholtz in Germany rank really well and above most leading universities in their field. As you say, these are not universities, but research institutes and therefore ignored by ARWU, US News and others.
IanCal 5y ago
Rankings can also be affected depending on how the identification of the university in the affiliation is done. Automated matches are often lower for languages that have different ways of writing things (often when there are different characters, or it's written in different languages), changes to location/name/structure, and how long it's been around. Based on a quick look at wikipedia: For Szeged, the university is kind of old but also only appeared in it's current form in 2000. It's also called Szegedi Tudományegyetem. It has multiple colleges inside it, so it may depend on how well those are represented in the data. Authors using acronyms more makes identification less likely, each of the colleges have acronyms. Finally, where authors publish makes a big difference - not all publishers open their data up and even those that allow access to some raw data it can be less reliable to extract. So if you publish in places where the publisher does not make the affiliations easily machine readable your work may be counted less in many of these automated rankings. Pubmed (not a publisher but a key way a lot of people get some of this data) didn't have affiliations for anyone but the first author until some fairly late date. Either automated mappings are used for a lot of work (since there are tens of millions of unique affiliation strings), or the source material is heavily curated (so for example only the "top" N journals are looked at). source: worked on affiliation mapping at Dimensions in a previous life