6 ms·

My favorite article on this topic is "Escaping science's paradox" by Stuart Buck[1]. I'm, in particular, interested by the idea (at least within the United Stat

by qchris 5y ago

My favorite article on this topic is "Escaping science's paradox" by Stuart Buck[1]. I'm, in particular, interested by the idea (at least within the United States) of "red teaming" science. This would involve having an independent agency funding attempts to replicate (and to find/publish flaws in) NSF- or NIH-funded projects, and publishing those. Ideally, the history of replication for authors' papers could then be part of the criteria for receiving funding for more novel research in the future. Obviously, there's a few fields where this might not work (you can't just create a second Large Hadron Collider for validation), but in areas from sociology to organic chemistry to environmental science, I think there's a lot of promise in that method for helping to re-align incentives around producing solid, replicable research. [1] https://www.worksinprogress.co/issue/escaping-sciences-paradox/ https://www.worksinprogress.co/issue/escaping-sciences-parad...

mherdeg 5y ago
I took a science journalism class in college where our instructor had us read a paper and then write the news story that explained what was interesting about the result. "You all got it wrong," he said, "the news is not that Amy Wagers could not make things work with mouse stem cells the way this prior paper said this one time. The news is that Wagers-ize is becoming a verb which means 'to disprove an amazing result after attempting to replicate it'. The lab has Wageres-ed another pluripotent stem cell result. The news is about how often this happens and what it means for this kind of science." This class was in 2006 and this later profile in 2008 seemed to bear things out the way he said: https://news.harvard.edu/gazette/story/2008/07/amy-wagers-focusing-on-stem-cell-biology/ https://news.harvard.edu/gazette/story/2008/07/amy-wagers-fo...
0x0203 5y ago
One thing I'd like to see is a requirement that for all government funded research, a certain percentage of that funding, say 30%, must go toward replicating other publicly funded research that has had less than 2 independent and non-affiliated labs replicate. Any original research couldn't be published until at least two independent and non-affiliated labs replicate based on the submitted paper and report on the results that can then be included with the original research. I'd like to see this across all of academia, but I imagine there are enough challenges with enforcing this in a productive manner already that doing it across all research becomes both impractical and difficult to prevent abuse. But at least with public funds, it would be nice to put in some checks to reduce the amount of fraudulent or sloppy research that tax payers pay for.
- the_snooze 5y ago
  I should point out that the notion of "replication" can often be way more difficult and nuanced than people expect. For one, what is the scope of the replication? Would it be simply to re-run the analysis on the data and make sure the math checks out? Or would it be to re-collect the data according to the methods described by the original researchers? The former is pretty easy, but only catches errors in the analysis phase (i.e., the data itself could be flawed). The latter is very comprehensive, but you essentially have to double up the effort on re-doing the study---which may not always be possible if you're studying a moving target (e.g., how the original SARS-CoV-2 variant spread through the initial set of hosts).
  - epgui 5y ago
    Here's an even easier set of requirement to simplify the first case: - Require all research to publish their source code. - Require all research to publish their raw data minus "PII". * Note: I use "PII" here with the intention of it taking the most liberal meaning possible, where privacy trumps transparency absolutely and where de-anonymization is impossible. This would rule out a lot of data, and personally I think we could take a more balanced approach, but even this minimalist approach would be a vast improvement on the current situation.
    - bjelkeman-again 5y ago
      When I learned at university that not all published research, especially government funded, didn’t do this already I was dumbfounded.
      - epgui 5y ago
        "Not all" is a big understatement... I would estimate that less than 0.00001% of published research does this. Every time I talk about this to someone (colleagues in adjacent fields, PIs...), they seem to give zero pucks. It's really mind-boggling.
  - OrderlyTiamat 5y ago
bee_rider 5y ago
Being on the science red team could also be really cool and fun. Since the goal is to explore the type of error or lie that gets through reliably, put new scientists on a team with some old greybeard, let's pass along that hard earned "how to screw up cleverly" experience.
- JacobThreeThree 5y ago
  >Being on the science red team could also be really cool and fun. I think it depends on what you're investigating, and how much is at stake. I doubt it would be much fun to be put on a corporate hit list. >The court was told that James Fries, professor of medicine at Stanford University, wrote to the then Merck head Ray Gilmartin in October 2000 to complain about the treatment of some of his researchers who had criticised the drug. >"Even worse were allegations of Merck damage control by intimidation," he wrote, ... "This has happened to at least eight (clinical) investigators ... I suppose I was mildly threatened myself but I never have spoken or written on these issues." https://www.cbsnews.com/news/merck-created-hit-list-to-destroy-neutralize-or-discredit-dissenting-doctors/ https://www.cbsnews.com/news/merck-created-hit-list-to-destr...
- mike_hearn 5y ago
  Talk to people who have actually done it. Not one will tell you it's cool or fun. Here's how science red teaming actually goes: 1. You download a paper and read it. It's got major, obvious problems that look suspiciously like they might be deliberate. 2. You report the problems to the authors. They never reply. 3. You report the problems to the journals. They never reply. 4. You report the problems to the university where those people work. They never reply. 5. Months have passed, you're tired of this and besides by now the same team has published 3 more papers all of which are also flawed. So you start hunting around for people who will reply, and eventually you find some people who run websites where bad science is discussed. They do reply and even publish an article you wrote about what is going wrong in science, but it's the wrong sort of site so nobody who can do anything about the problem is reading it. 6. In addition if you red-teamed the wrong field, you get booted off Twitter for "spreading misinformation" and the press describe you as a right wing science denier. Nobody has ever asked you what your politics are and you're not denying science, you're denying pseudo-science in an effort to make actual science better, but none of that matters. 7. You realize that this is a pointless endeavour. The people you hoped would welcome your "red teaming" are actually committed to defending the institutions regardless of merit, and the people who actually do welcome it are all ideologically persona non grata in the academic world - even inviting them to give a talk risks your cancellation. The End. An essay that explores this problem from the perspective of psychology reform can be found here: https://www.psychologytoday.com/us/blog/how-do-you-know/202106/the-science-reform-brain-drain https://www.psychologytoday.com/us/blog/how-do-you-know/2021...
a-dub 5y ago
it's the same as everything. there should be more and easier money for the less rewarding task of verification/replication. some people actually enjoy this sort of work just as much as some people enjoy being on the bleeding edge... but there are probably less of them. where it would get complicated is also the same as everything. when the verification effort neither supports nor refutes the original one. many would argue that it means it wasn't done right, but lots of things aren't done right in life. then there can be the triple replication revolution! so it goes...
- qchris 5y ago
  This isn't intended as snarky, but I don't understand what "it's the same as everything" is supposed to mean. What is "it"? What is "everything"? Why are they the same? I'd also argue that your reduction of this problem sort of misses the point. One of the big problems with the way that studies are done is not that replication efforts aren't conclusive (it's very difficult to prove something doesn't exist), it's that a) non-replicable studies are generally considered as valuable as replicable ones, and b) as a result, it's extremely difficult to replicate many studies to begin with, because there's no incentive to take the time to make it possible. Even if the end result of a replication paper is "we couldn't produce the same results", the people working on it can say "this author's experiments were exceedingly difficult to even try to reproduce," or conversely "we didn't get the same results, but their data collection methods and analysis code were well-documented and accessible." That has a lot of value! If you tried doing triple replication for every paper, I agree that maybe wouldn't be the best use of resources. But the current state of affairs is so bad that a well-organized drive to create single-attempt replication on a fraction of publicly-funded projects has the potential to be a significant driver of change.
  - a-dub 5y ago
    "the same as everything" is an observation that often times verification/correctness/accuracy efforts are tossed aside in favor of new development and this is a truism across many fields. in science you see this as funding being committed to shiny new nature and science cover stories, with replication being left as an afterthought. in software you see this as heavy commitments to new features that drive revenue, with security/compliance/architecture and qa remaining underfunded and less respected. (until, of course, the problems that result from underfunding them make themselves apparent).
Beldin 5y ago
One problem is that the amount of scientific output is increasing at an increasing rate. This means that the vast, vary majority of works will never be considered for replication - even with a dedicated replication institute. So for most applicants, the amount of replicated results will be 0.

by qchris 5y ago

mherdeg 5y ago

I took a science journalism class in college where our instructor had us read a paper and then write the news story that explained what was interesting about the result. "You all got it wrong," he said, "the news is not that Amy Wagers could not make things work with mouse stem cells the way this prior paper said this one time. The news is that Wagers-ize is becoming a verb which means 'to disprove an amazing result after attempting to replicate it'. The lab has Wageres-ed another pluripotent stem cell result. The news is about how often this happens and what it means for this kind of science." This class was in 2006 and this later profile in 2008 seemed to bear things out the way he said: https://news.harvard.edu/gazette/story/2008/07/amy-wagers-focusing-on-stem-cell-biology/ https://news.harvard.edu/gazette/story/2008/07/amy-wagers-fo...

0x0203 5y ago

One thing I'd like to see is a requirement that for all government funded research, a certain percentage of that funding, say 30%, must go toward replicating other publicly funded research that has had less than 2 independent and non-affiliated labs replicate. Any original research couldn't be published until at least two independent and non-affiliated labs replicate based on the submitted paper and report on the results that can then be included with the original research. I'd like to see this across all of academia, but I imagine there are enough challenges with enforcing this in a productive manner already that doing it across all research becomes both impractical and difficult to prevent abuse. But at least with public funds, it would be nice to put in some checks to reduce the amount of fraudulent or sloppy research that tax payers pay for.

the_snooze 5y ago
I should point out that the notion of "replication" can often be way more difficult and nuanced than people expect. For one, what is the scope of the replication? Would it be simply to re-run the analysis on the data and make sure the math checks out? Or would it be to re-collect the data according to the methods described by the original researchers? The former is pretty easy, but only catches errors in the analysis phase (i.e., the data itself could be flawed). The latter is very comprehensive, but you essentially have to double up the effort on re-doing the study---which may not always be possible if you're studying a moving target (e.g., how the original SARS-CoV-2 variant spread through the initial set of hosts).
- epgui 5y ago
  Here's an even easier set of requirement to simplify the first case: - Require all research to publish their source code. - Require all research to publish their raw data minus "PII". * Note: I use "PII" here with the intention of it taking the most liberal meaning possible, where privacy trumps transparency absolutely and where de-anonymization is impossible. This would rule out a lot of data, and personally I think we could take a more balanced approach, but even this minimalist approach would be a vast improvement on the current situation.
  - bjelkeman-again 5y ago
    When I learned at university that not all published research, especially government funded, didn’t do this already I was dumbfounded.
    - epgui 5y ago
      "Not all" is a big understatement... I would estimate that less than 0.00001% of published research does this. Every time I talk about this to someone (colleagues in adjacent fields, PIs...), they seem to give zero pucks. It's really mind-boggling.
- OrderlyTiamat 5y ago

bee_rider 5y ago

Being on the science red team could also be really cool and fun. Since the goal is to explore the type of error or lie that gets through reliably, put new scientists on a team with some old greybeard, let's pass along that hard earned "how to screw up cleverly" experience.

JacobThreeThree 5y ago
>Being on the science red team could also be really cool and fun. I think it depends on what you're investigating, and how much is at stake. I doubt it would be much fun to be put on a corporate hit list. >The court was told that James Fries, professor of medicine at Stanford University, wrote to the then Merck head Ray Gilmartin in October 2000 to complain about the treatment of some of his researchers who had criticised the drug. >"Even worse were allegations of Merck damage control by intimidation," he wrote, ... "This has happened to at least eight (clinical) investigators ... I suppose I was mildly threatened myself but I never have spoken or written on these issues." https://www.cbsnews.com/news/merck-created-hit-list-to-destroy-neutralize-or-discredit-dissenting-doctors/ https://www.cbsnews.com/news/merck-created-hit-list-to-destr...
mike_hearn 5y ago
Talk to people who have actually done it. Not one will tell you it's cool or fun. Here's how science red teaming actually goes: 1. You download a paper and read it. It's got major, obvious problems that look suspiciously like they might be deliberate. 2. You report the problems to the authors. They never reply. 3. You report the problems to the journals. They never reply. 4. You report the problems to the university where those people work. They never reply. 5. Months have passed, you're tired of this and besides by now the same team has published 3 more papers all of which are also flawed. So you start hunting around for people who will reply, and eventually you find some people who run websites where bad science is discussed. They do reply and even publish an article you wrote about what is going wrong in science, but it's the wrong sort of site so nobody who can do anything about the problem is reading it. 6. In addition if you red-teamed the wrong field, you get booted off Twitter for "spreading misinformation" and the press describe you as a right wing science denier. Nobody has ever asked you what your politics are and you're not denying science, you're denying pseudo-science in an effort to make actual science better, but none of that matters. 7. You realize that this is a pointless endeavour. The people you hoped would welcome your "red teaming" are actually committed to defending the institutions regardless of merit, and the people who actually do welcome it are all ideologically persona non grata in the academic world - even inviting them to give a talk risks your cancellation. The End. An essay that explores this problem from the perspective of psychology reform can be found here: https://www.psychologytoday.com/us/blog/how-do-you-know/202106/the-science-reform-brain-drain https://www.psychologytoday.com/us/blog/how-do-you-know/2021...

a-dub 5y ago

it's the same as everything. there should be more and easier money for the less rewarding task of verification/replication. some people actually enjoy this sort of work just as much as some people enjoy being on the bleeding edge... but there are probably less of them. where it would get complicated is also the same as everything. when the verification effort neither supports nor refutes the original one. many would argue that it means it wasn't done right, but lots of things aren't done right in life. then there can be the triple replication revolution! so it goes...

qchris 5y ago
This isn't intended as snarky, but I don't understand what "it's the same as everything" is supposed to mean. What is "it"? What is "everything"? Why are they the same? I'd also argue that your reduction of this problem sort of misses the point. One of the big problems with the way that studies are done is not that replication efforts aren't conclusive (it's very difficult to prove something doesn't exist), it's that a) non-replicable studies are generally considered as valuable as replicable ones, and b) as a result, it's extremely difficult to replicate many studies to begin with, because there's no incentive to take the time to make it possible. Even if the end result of a replication paper is "we couldn't produce the same results", the people working on it can say "this author's experiments were exceedingly difficult to even try to reproduce," or conversely "we didn't get the same results, but their data collection methods and analysis code were well-documented and accessible." That has a lot of value! If you tried doing triple replication for every paper, I agree that maybe wouldn't be the best use of resources. But the current state of affairs is so bad that a well-organized drive to create single-attempt replication on a fraction of publicly-funded projects has the potential to be a significant driver of change.
- a-dub 5y ago
  "the same as everything" is an observation that often times verification/correctness/accuracy efforts are tossed aside in favor of new development and this is a truism across many fields. in science you see this as funding being committed to shiny new nature and science cover stories, with replication being left as an afterthought. in software you see this as heavy commitments to new features that drive revenue, with security/compliance/architecture and qa remaining underfunded and less respected. (until, of course, the problems that result from underfunding them make themselves apparent).

Beldin 5y ago

One problem is that the amount of scientific output is increasing at an increasing rate. This means that the vast, vary majority of works will never be considered for replication - even with a dedicated replication institute. So for most applicants, the amount of replicated results will be 0.