3 ms·
When something becomes widely known as a proxy to measure an effect, it ceases to be a useful measure, because people begin to consciously influence it. It's a
by nine_k 6y ago
When something becomes widely known as a proxy to measure an effect, it ceases to be a useful measure, because people begin to consciously influence it. It's a well-known thing.
When scientific results become a proxy to the correctness of a political agenda (which is often a business interest in disguise), they become less and less scientific.
It's a perfect catch-22.
Same sadly applies to the internal tools of science: impact index, citation index, discovery of a novel effect, etc. There is a lot of incentives to game these metrics, and gaming them is, well, not unheard of.
In applied research there is more of a reality check: if the device based on your research obviously does not work, it cannot be produced and sold, so you want your research to reflect reality as best you can. But more subtle things, like public policy, lack this immediate feedback loop. So there is a large hazard for a researcher, when asked: "what is the value of 2 × 2 ?" to ask back: "are we selling or buying?" and find a plausible "right" answer that supports a desired case.
But, unlike in elementary arithmetics, nobody knows a certain right answer in many areas of natural science, so an honest mistake is hard to tell from a... less honest one. There must be a colossal pressure to exploit this in a situation of a high-stakes political choice.
I have no idea how to solve these problems. But at least the public should be aware that these problems exist.
- XargonEnder 6y agoI agree with you and I want to add one more layer. I program analytics for a living. I often find that people fudge numbers without even being conscious of it. If I know my boss wants the numbers on this report to be high, I'll unconsciously choose an algorithm that gives higher numbers than one that gives lower numbers. Even if both algorithms are "reasonable".