3 ms·

> People aren’t in academia for the money. It’s a vocation. You’re not going to have many people milking the system. Before introducing the KPIs, a majority of

> People aren’t in academia for the money. It’s a vocation. You’re not going to have many people milking the system. Before introducing the KPIs, a majority of polish science was basically people milking the system and doing barely any (valueable) research. It was seen as an easy, safe and ok paying job where the only major hassle is having to teach the students. You often needed connections to get in. It was partially like that because of the communist legacy, where playing ball with the communist party was the most important merit for promotion, which, over the course of 45 years (the span of communism in Poland), filled the academia management ranks with conformist mediocrities. Now, after a series of major reforms, there's a ton of KPIs, and people are now doing plenty of makework research to collect the required points, but still little valueable work gets done. Also, people interested in doing genuine science who would be doing it under the old system are now discouraged from joining academia, because in the system they're expected to game the points system and not to do real work. What is the lesson from this is? Creating institutionalized science is hard? It requires a long tradition and scientific cultural standards and can't be just wished into place by bureaucrats? Also, perhaps it's good to be doing the science for some purpose, which in the US case are often DoD grants, where the military expects some practical application. This application may be extremely distant, vague and uncertain (they fund pure math research!), but still, they're the client and they expect results. Whereas the (unstated) goal of science in Poland seems to be just to increase the prestige of Polish science and its Universities by getting papers into prestigious journals, whereas the actual science being done doesn't matter at all - basically state-level navel gazing.

lkozma 5y ago
I recognize the symptoms you describe, but I find the picture you painted a bit too pessimistic, given that in the field that I'm familiar with (theoretical CS), groups in Poland have been among the strongest and most visible in Europe in the past 10 years.
- ed_balls 5y ago
  because it's fresh blood and there is a skill set filter which reduces nepotism and other forms of corruption. Looking at other departments, say, sociology* it's a dumpster. Full of cronies, retired politicians and relatives. *Just an example which doesn't require "hard" skills like math.
MayeulC 5y ago
I've long wondered if the answer for this and other issues like tax evasion wouldn't be to often change the metrics. If you start cheating the metrics, or optimizing a lot towards them, it becomes counter-productive when they change. As such, the most efficient way forward would be to work without trying to optimize for a temporary metric. On the flip side, it would be troublesome to convince people to different, complex forms each time. What first gave me the idea was the concept of "lubricating" headers (submitting some with random values) for future http protocols, to combat "ossification", where middle boxes start to meddle with them and become obsolete when they don't recognize the new fields, instead of transmitting them.
- tchalla 5y ago
  A slightly different take from me would be to have multiple metrics that kind of cross each other out making the tradeoffs explicit instead of implicit. It's fine to temporarily optimise towards anyone but at least we know which trade off was made.
- ByteJockey 5y ago
  I always thought having a pool of 20 metrics and randomly picking 4 every quarter/year for bonuses/promotions/whatever might get people to stop optimizing for single thing.
  - bongoman37 5y ago
    This is fine until someone sues because they had similar or better metrics than someone else but the other person got in in one year and they were refused. A lot of the reliance on metrics is CYA by administrators.
    - MaxBarraclough 5y ago
      A more deterministic alternative: a weighted sum of the pool of metrics, where the weight can be gradually adjusted over time.
  - Beldin 5y ago
aristophenes 5y ago
This is a good cautionary lesson. Problems like collusion rings in computer science are more serious than they appear. If left unchecked for a decade or two, the cheaters will rise to the top and take the positions of power in the field, and then it's going to be almost impossible to fix the system. Relying on altruistic tendencies for people in academia is not adequate. Everyone starts out in academics as school children, and get filtered out or filter themselves out pursuing other things. Those who remain will be the ones who love to learn and teach, those who just cannot accept loss/failure, and, sadly, those who are afraid of change. The more competitive the field becomes, the harder it is to succeed, the more we select for the hyper-competitive or fearful over the altruistic.
- marcus_holmes 5y ago
  I looked at doing my PhD. I was really interested in startup failure and wanted to research that. The system I was presented with would have meant my supervisors got more say in what I spent my time on than I did. They heavily skewed to supporting existing psychological models of entrepreneurial failure which I wasn't interested in. The bureaucracy, authoritarianism and endless hoop-jumping around it was a total red flag for me. I opted out. My conclusions: PhD grads aren't smarter than others. They're just more willing to put up with bullshit, conform to meaningless rules, and jump through hoops. Academic research is rarely about the things it says that it's about, and seems to be more about maintaining/improving the career prospects of the academics. Academic research is heavily affected by political spats with other academics that have nothing to do with the actual subject, but more to do with ego, pride and interpersonal dislike.
  - jasonwatkinspdx 5y ago
    My brother abandoned his PhD at CMU just before his defense (which would have gone fine). He was intensely passionate about his topic area, logical frameworks, which is very academic stuff indeed. Not really something you could pursue in industry. As he was wrapping up his PhD he also got his top choice for a post doc position at Inria, but the amount of political maneuvering he had to do to get it made him so angry he abandoned his dream of staying in academia. He just couldn't stomach the idea of having to do that politicking over and over again all his life. So he went to a hedge fund to work a little bit more than a decade, got his magic number of millions, and is a full time dad now. Humanity as a whole is losing out on some potentially amazing long term research and researchers because of this dynamic. I have no idea how to fix it systematically.
khafra 5y ago
The old system had too much Slack. The new system has too much Moloch. https://slatestarcodex.com/2020/05/12/studies-on-slack/ https://slatestarcodex.com/2020/05/12/studies-on-slack/

lkozma 5y ago

I recognize the symptoms you describe, but I find the picture you painted a bit too pessimistic, given that in the field that I'm familiar with (theoretical CS), groups in Poland have been among the strongest and most visible in Europe in the past 10 years.

ed_balls 5y ago
because it's fresh blood and there is a skill set filter which reduces nepotism and other forms of corruption. Looking at other departments, say, sociology* it's a dumpster. Full of cronies, retired politicians and relatives. *Just an example which doesn't require "hard" skills like math.

MayeulC 5y ago

I've long wondered if the answer for this and other issues like tax evasion wouldn't be to often change the metrics. If you start cheating the metrics, or optimizing a lot towards them, it becomes counter-productive when they change. As such, the most efficient way forward would be to work without trying to optimize for a temporary metric. On the flip side, it would be troublesome to convince people to different, complex forms each time. What first gave me the idea was the concept of "lubricating" headers (submitting some with random values) for future http protocols, to combat "ossification", where middle boxes start to meddle with them and become obsolete when they don't recognize the new fields, instead of transmitting them.

tchalla 5y ago
A slightly different take from me would be to have multiple metrics that kind of cross each other out making the tradeoffs explicit instead of implicit. It's fine to temporarily optimise towards anyone but at least we know which trade off was made.
ByteJockey 5y ago
I always thought having a pool of 20 metrics and randomly picking 4 every quarter/year for bonuses/promotions/whatever might get people to stop optimizing for single thing.
- bongoman37 5y ago
  This is fine until someone sues because they had similar or better metrics than someone else but the other person got in in one year and they were refused. A lot of the reliance on metrics is CYA by administrators.
  - MaxBarraclough 5y ago
    A more deterministic alternative: a weighted sum of the pool of metrics, where the weight can be gradually adjusted over time.
- Beldin 5y ago

aristophenes 5y ago

This is a good cautionary lesson. Problems like collusion rings in computer science are more serious than they appear. If left unchecked for a decade or two, the cheaters will rise to the top and take the positions of power in the field, and then it's going to be almost impossible to fix the system. Relying on altruistic tendencies for people in academia is not adequate. Everyone starts out in academics as school children, and get filtered out or filter themselves out pursuing other things. Those who remain will be the ones who love to learn and teach, those who just cannot accept loss/failure, and, sadly, those who are afraid of change. The more competitive the field becomes, the harder it is to succeed, the more we select for the hyper-competitive or fearful over the altruistic.

marcus_holmes 5y ago
I looked at doing my PhD. I was really interested in startup failure and wanted to research that. The system I was presented with would have meant my supervisors got more say in what I spent my time on than I did. They heavily skewed to supporting existing psychological models of entrepreneurial failure which I wasn't interested in. The bureaucracy, authoritarianism and endless hoop-jumping around it was a total red flag for me. I opted out. My conclusions: PhD grads aren't smarter than others. They're just more willing to put up with bullshit, conform to meaningless rules, and jump through hoops. Academic research is rarely about the things it says that it's about, and seems to be more about maintaining/improving the career prospects of the academics. Academic research is heavily affected by political spats with other academics that have nothing to do with the actual subject, but more to do with ego, pride and interpersonal dislike.
- jasonwatkinspdx 5y ago
  My brother abandoned his PhD at CMU just before his defense (which would have gone fine). He was intensely passionate about his topic area, logical frameworks, which is very academic stuff indeed. Not really something you could pursue in industry. As he was wrapping up his PhD he also got his top choice for a post doc position at Inria, but the amount of political maneuvering he had to do to get it made him so angry he abandoned his dream of staying in academia. He just couldn't stomach the idea of having to do that politicking over and over again all his life. So he went to a hedge fund to work a little bit more than a decade, got his magic number of millions, and is a full time dad now. Humanity as a whole is losing out on some potentially amazing long term research and researchers because of this dynamic. I have no idea how to fix it systematically.

khafra 5y ago

The old system had too much Slack. The new system has too much Moloch. https://slatestarcodex.com/2020/05/12/studies-on-slack/ https://slatestarcodex.com/2020/05/12/studies-on-slack/