2 ms·
My view is that, like many other fields (including notably software engineering) before it, Academia has fallen victim to Goodhart/Campbell's law. Goodhart's l
by tpoacher 5y ago
My view is that, like many other fields (including notably software engineering) before it, Academia has fallen victim to Goodhart/Campbell's law.
Goodhart's law is an adage named after economist Charles Goodhart: "When a measure becomes a target, it ceases to be a good measure."
This follows from individuals trying to anticipate the effect of a policy and then taking actions which alter its outcome.
Campbell's law (by Donald T. Campbell, a psychologist and social scientist), is similar, but has a more concrete focus on the predictably negative unintended consequences of using such indicators for decision / policy making: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
For example, how schools evaluated by marks end up 'teaching to the test' or outright help children cheat.
A simple combination of the two (to misuse Donald trump style) basically would say that using metrics: sounds good, doesn't work; worse still, it backfires. Goodhart's law focuses on the fact that metrics don't work; Campbell's on the fact that they tend to backfire.
We have seen this time and time again in software engineering, when managers try to use crap like LOC metrics, or more recently "slack activity" to judge the quality of a software engineer.
Academia is now experiencing its own version of Goodhart/Campbell's law. Between impact factors, h-indices, and now REF exercises, scientific progress is but an afterthought, and the system self-selects either those with a high ability of navigating this monstrous maze of inane metrics created by bureaucrats, or those with the ability to successfully commit academic dishonesty (p-hacking, optimal division of publication units etc) without getting caught, or both. And only extremely occasionally, people with truly novel ideas and output, which happen to somehow still manage to obtain funding despite not fitting into any of the tight little checkboxes that need to be ticked to get a grant on the latest grant-bandwagon.
- blt 5y agoWhat is a REF exercise?
- tpoacher 5y agoAh it's a UK thing. Not sure what kind of equivalents other countries have, but I'm sure they do in similar forms. It stands for Research Excellence Framework, and effectively it's a metric used by government and research councils to allocate funding to more 'deserving' universities / groups, by rewarding departments with "high quality paper outputs". This has led to effectively departments launching tedious bureaucratic exercises and workshops where all members of staff need to submit summaries for "star-ratings" for departmental sifting, submissions of only certain kinds of papers to particular journals, eschewing both high-risk and low-hanging-fruit research in favour of controversial or 'popular' topics which are more likely to gather citations, "buying" researchers for their paper co-authorships within the currently valid REF-cycle rather than based on their research/teaching skills or interests, etc.