5 ms·

No, this is untrue. Metric hacking doesn't really exist in alternative systems of doing science. For example, it didn't exist in the Eastern Europe during commu

by js8 3y ago

No, this is untrue. Metric hacking doesn't really exist in alternative systems of doing science. For example, it didn't exist in the Eastern Europe during communism. There, being an authority in research was more based on nepotism than on merit, but to a large extent even fringe people were allowed to pursue research. It is also not a feature of a tenure system, which existed in the West during the Cold War. The idea that we need to continuously monitor people for performance (measure them), rather than (for example) accept them into a community and just let them evolve there (for better or worse), is very intimately tied to capitalism. Also, all the KPI systems based on objective metric ignore a really simple metric, that you can simply ask every scientist an opinion (for example, who are the leading people in your field) and use that as a metric, as a recommendation what kind of research is to be financed more and what kind is to be financed less. Again, this is because capitalism cannot trust employees by default, it has to put some other authority in charge of them.

godelski 3y ago
> Metric hacking doesn't really exist in alternative systems of doing science. For example, it didn't exist in the Eastern Europe during communism. I'd like to make you aware of this little known thing called The 5 Year Plans.
- js8 3y ago
  I was talking about science, not planned economy in general. Planned economy is problematic for many of the same reasons for which capitalist production in large corporations is problematic. (And capitalist system which wouldn't have many large hierarchically controlled corporations doesn't exist.) My point is, there existed different systems of doing science, which weren't measured as much (to the extent that would cause overproduction of articles that no scientist has time to read), because they had different preference than "scientific efficacy by an objective metric". For example, preference for somebody being authority in a scientific field for life. Or preference for academia to be a place for outcasts. Or preference for democratic decision making about what to scientifically pursue. These are all non-capitalist preferences.
  - godelski 3y ago
    > I was talking about science, not planned economy in general. I was talking about metric hacking, which isn't exclusive to science. I could have been more explicit though. But I still think you unnecessarily brought in a red herring by bringing up economic systems and I could have responded more appropriately. But our discussion is off the rails now. > My point is, there existed different systems of doing science, which weren't measured as much Science has always been about measurements and philosophy. It is the balance. The issue is about the over metrification as to what the science means. You're definitely right to point out these issues within capitalist frameworks. You'll find many of a comment from me complaining about how we do this within corporations as well. KPIs rule everything even when they mean nothing. Quarterly profits over long term profits. Yes, these are metric hacking and incredibly short sighted things. But at the same time it would be naive to assume that there are not parallels under communistic rule[0]. There are plenty of examples as there are under all the systems that I mentioned. The problem here is that you're willing to look at flaws in a system you disagree with (which you may notice no one is claiming the capitalism is flawless nor are many even countering your critiques. Pay close attention), the problem is that you're unwilling to look at the flaws within your preferred system. There are no globally optimal solutions to problems such as these, and thus there is no flawless system. The irony of this is that by ignoring such flaws you're actually performing the very metric hacking we are complaining about and it would not be unreasonable to call this hypocritical. Think about comedians if we need an analogy: you can't make fun of others if you aren't willing to make fun of yourself even more. With belief systems it is similarly more important that you are more critical of the ones you prescribe to than the ones you disagree with. After all, don't you want to improve upon them? We can have discussions of which is better and which is worse but that needs context and honesty about the downsides of each system. But no discussion can exist if you're unwilling to admit to flaws. [0] It's hard to give a specific example because the Scottsman card has already been played elsewhere. A difficult part of conversations in this whole abstract "communism v capitalism" is that no one agrees on the fucking fundamental definitions and so arguments are abound. IMO the capitalists seem to have lower variance but I digress. For an example, if you are one of the people that considers modern China communist then we can point to them. They're doing the same shit as we are in the US in terms of papers, but I'd honestly say worse simply because there's more of them doing it. If you don't think China is communist, then well I'm not sure we can really make comparisons if no true communist regime has existed and it's probably not worth talking about "true communism" if it is so vulnerable that every attempt has failed within a few decades (sorry, can't just blame it on the CIA. They play a role and did fucked up shit but they aren't wizards either).
woolion 3y ago
>Metric hacking doesn't really exist in alternative systems of doing science. For example, it didn't exist in the Eastern Europe during communism. Sure, metric hacking is not a problem when the crux of the matter is actually bending truth to follow the party line or face repercussions. Repercussions that could be years of imprisonment up to death penalty. A cursory research on "scientific research under communism" yields [0], which seems to line up with the experience from people coming from former communist countries (including Mao's China and some former Eastern European countries). I am also reminded of Luzin [1]. The previous article does not even mention pure mathematics as falling under repression, but even this domain (which was a strong point of Soviet Russia) could send researchers into concentration camps for dubious political motives. So yes, metric hacking does not matter when the whole thing is bogus. [0]: https://en.wikipedia.org/wiki/Repression_of_science_in_the_Soviet_Union https://en.wikipedia.org/wiki/Repression_of_science_in_the_S... [1]: https://en.wikipedia.org/wiki/Nikolai_Luzin#Luzin_affair_of_1936 https://en.wikipedia.org/wiki/Nikolai_Luzin#Luzin_affair_of_...
- js8 3y ago
  My father was a university researcher/teacher (assistant professor) in Czechoslovakia, during communist regime. It's not as black and white as these links might lead you to. He was never higher ranking (didn't get professorship) until the 1989 revolution, AFAICT he was more interested in research (thermodynamics) than career. During real socialism, academia was often a refuge of smart and freethinking people, if you were able to get there (some didn't for political reasons). It wasn't a highly paid job. And it also was very inefficient. There were completely useless departments, and the department heads were political positions. But some good science got done, without having to toe the party line, and without having to bend metrics and having to publish lots of repeated nonsense. (I am not advocating that particular system, I think it was worse, but I am pointing out that measurement is not inevitable.) One result of liberal (capitalist) approach to science is that people get really good at marketing themselves. That's why science in the Eastern Europe often gets bad optics, because that self-promoting aspect of it wasn't as prevalent there.
  - woolion 3y ago

by js8 3y ago

godelski 3y ago

> Metric hacking doesn't really exist in alternative systems of doing science. For example, it didn't exist in the Eastern Europe during communism. I'd like to make you aware of this little known thing called The 5 Year Plans.

js8 3y ago
I was talking about science, not planned economy in general. Planned economy is problematic for many of the same reasons for which capitalist production in large corporations is problematic. (And capitalist system which wouldn't have many large hierarchically controlled corporations doesn't exist.) My point is, there existed different systems of doing science, which weren't measured as much (to the extent that would cause overproduction of articles that no scientist has time to read), because they had different preference than "scientific efficacy by an objective metric". For example, preference for somebody being authority in a scientific field for life. Or preference for academia to be a place for outcasts. Or preference for democratic decision making about what to scientifically pursue. These are all non-capitalist preferences.
- godelski 3y ago
  > I was talking about science, not planned economy in general. I was talking about metric hacking, which isn't exclusive to science. I could have been more explicit though. But I still think you unnecessarily brought in a red herring by bringing up economic systems and I could have responded more appropriately. But our discussion is off the rails now. > My point is, there existed different systems of doing science, which weren't measured as much Science has always been about measurements and philosophy. It is the balance. The issue is about the over metrification as to what the science means. You're definitely right to point out these issues within capitalist frameworks. You'll find many of a comment from me complaining about how we do this within corporations as well. KPIs rule everything even when they mean nothing. Quarterly profits over long term profits. Yes, these are metric hacking and incredibly short sighted things. But at the same time it would be naive to assume that there are not parallels under communistic rule[0]. There are plenty of examples as there are under all the systems that I mentioned. The problem here is that you're willing to look at flaws in a system you disagree with (which you may notice no one is claiming the capitalism is flawless nor are many even countering your critiques. Pay close attention), the problem is that you're unwilling to look at the flaws within your preferred system. There are no globally optimal solutions to problems such as these, and thus there is no flawless system. The irony of this is that by ignoring such flaws you're actually performing the very metric hacking we are complaining about and it would not be unreasonable to call this hypocritical. Think about comedians if we need an analogy: you can't make fun of others if you aren't willing to make fun of yourself even more. With belief systems it is similarly more important that you are more critical of the ones you prescribe to than the ones you disagree with. After all, don't you want to improve upon them? We can have discussions of which is better and which is worse but that needs context and honesty about the downsides of each system. But no discussion can exist if you're unwilling to admit to flaws. [0] It's hard to give a specific example because the Scottsman card has already been played elsewhere. A difficult part of conversations in this whole abstract "communism v capitalism" is that no one agrees on the fucking fundamental definitions and so arguments are abound. IMO the capitalists seem to have lower variance but I digress. For an example, if you are one of the people that considers modern China communist then we can point to them. They're doing the same shit as we are in the US in terms of papers, but I'd honestly say worse simply because there's more of them doing it. If you don't think China is communist, then well I'm not sure we can really make comparisons if no true communist regime has existed and it's probably not worth talking about "true communism" if it is so vulnerable that every attempt has failed within a few decades (sorry, can't just blame it on the CIA. They play a role and did fucked up shit but they aren't wizards either).

woolion 3y ago

>Metric hacking doesn't really exist in alternative systems of doing science. For example, it didn't exist in the Eastern Europe during communism. Sure, metric hacking is not a problem when the crux of the matter is actually bending truth to follow the party line or face repercussions. Repercussions that could be years of imprisonment up to death penalty. A cursory research on "scientific research under communism" yields [0], which seems to line up with the experience from people coming from former communist countries (including Mao's China and some former Eastern European countries). I am also reminded of Luzin [1]. The previous article does not even mention pure mathematics as falling under repression, but even this domain (which was a strong point of Soviet Russia) could send researchers into concentration camps for dubious political motives. So yes, metric hacking does not matter when the whole thing is bogus. [0]: https://en.wikipedia.org/wiki/Repression_of_science_in_the_Soviet_Union https://en.wikipedia.org/wiki/Repression_of_science_in_the_S... [1]: https://en.wikipedia.org/wiki/Nikolai_Luzin#Luzin_affair_of_1936 https://en.wikipedia.org/wiki/Nikolai_Luzin#Luzin_affair_of_...

js8 3y ago
My father was a university researcher/teacher (assistant professor) in Czechoslovakia, during communist regime. It's not as black and white as these links might lead you to. He was never higher ranking (didn't get professorship) until the 1989 revolution, AFAICT he was more interested in research (thermodynamics) than career. During real socialism, academia was often a refuge of smart and freethinking people, if you were able to get there (some didn't for political reasons). It wasn't a highly paid job. And it also was very inefficient. There were completely useless departments, and the department heads were political positions. But some good science got done, without having to toe the party line, and without having to bend metrics and having to publish lots of repeated nonsense. (I am not advocating that particular system, I think it was worse, but I am pointing out that measurement is not inevitable.) One result of liberal (capitalist) approach to science is that people get really good at marketing themselves. That's why science in the Eastern Europe often gets bad optics, because that self-promoting aspect of it wasn't as prevalent there.
- woolion 3y ago