2 ms·
Talk to people who have actually done it. Not one will tell you it's cool or fun. Here's how science red teaming actually goes: 1. You download a paper and rea
by mike_hearn 5y ago
Talk to people who have actually done it. Not one will tell you it's cool or fun. Here's how science red teaming actually goes:
1. You download a paper and read it. It's got major, obvious problems that look suspiciously like they might be deliberate.
2. You report the problems to the authors. They never reply.
3. You report the problems to the journals. They never reply.
4. You report the problems to the university where those people work. They never reply.
5. Months have passed, you're tired of this and besides by now the same team has published 3 more papers all of which are also flawed. So you start hunting around for people who will reply, and eventually you find some people who run websites where bad science is discussed. They do reply and even publish an article you wrote about what is going wrong in science, but it's the wrong sort of site so nobody who can do anything about the problem is reading it.
6. In addition if you red-teamed the wrong field, you get booted off Twitter for "spreading misinformation" and the press describe you as a right wing science denier. Nobody has ever asked you what your politics are and you're not denying science, you're denying pseudo-science in an effort to make actual science better, but none of that matters.
7. You realize that this is a pointless endeavour. The people you hoped would welcome your "red teaming" are actually committed to defending the institutions regardless of merit, and the people who actually do welcome it are all ideologically persona non grata in the academic world - even inviting them to give a talk risks your cancellation. The End.
An essay that explores this problem from the perspective of psychology reform can be found here:
https://www.psychologytoday.com/us/blog/how-do-you-know/202106/the-science-reform-brain-drain https://www.psychologytoday.com/us/blog/how-do-you-know/2021...