5 ms·

Millions of dollars’ worth of research in limbo at NIH

79 points by endswapper 9y ago

epmaybe 9y ago
Whenever I read stories like these I tend to take it with a grain of salt. There is always another part of the story we are missing. It will however be interesting to follow the development of this. Hopefully Washington Post or another outlet follows up with this.
- xenophonf 9y ago
  Here's the backstory: http://www.sciencemag.org/news/2017/02/firing-veteran-nih-scientist-prompts-protests-over-publication-ban http://www.sciencemag.org/news/2017/02/firing-veteran-nih-sc... It sounds like these junior postdocs are trying to salvage their careers after getting seriously screwed over by Braun's misfeasance. That sucks but I'm not sure the data management issues are as trivial as they claim. Experimental protocols are designed to protect the safety of human subjects. Those protections exist because of past transgressions that lead to serious harm and suffering---remember the Tuskegee syphilis experiment? NIDCD is probably right to throw out the data if the protocol was violated for nearly half the study participants covered by their audit.
  - nonbel 9y ago
    True story, I once went to try out being a subject for one of these take a drug then do some tests with an EEG hooked up, etc kind of study. They never once mentioned it, but then in the informed consent form there was a whole section agreeing to a spinal tap. I was like wtf, and they said "sorry we gave you an old version on accident" then replaced the form with one without the spinal tap section. The study seemed legit so it probably was just an honest error, but I wonder how many people agreed to a spinal tap without realizing what it was.
    - Cass 9y ago
      Research/academia can certainly be kind of screwed up, but I doubt that any ethics board in a developed country is going to okay a study that involves tricking people into a spinal tap. And all the consent forms patients are asked to sign before agreeing to a procedure, the ones that tell you about the procedure and the risks in excruciating detail, are as much for the protection of the doctor as the patient. That way, if the patient ends up paralyzed, at least they can't claim you didn't warn them when the case goes to court. So unless you were dealing with a single rogue Mad Scientist, I'd assume that was an error.
      - nonbel 9y ago
        I dunno, csf samples (and corresponding data) can be quite valuable. For example, I know someone who had a spinal cyst removed and learned that to this day they do not understand if the fluid is coming from inside or outside the cyst. We lack basic mechanistic information about these issues because there is so little data (then again the papers I looked at that did have such data only checked averages and otherwise weak/incorrect analysis).
    - kevin_thibedeau 9y ago
      Maybe it was a sugar pill and they were just screening for who would read the fine print.
    - arkades 9y ago
      There isn't really any trickery here. If at some point they were to come at you with a needle and you said, "uh, I didn't agree to that," that's where it would end. Consent forms for treatment are only valid until you withdraw consent.
      - nonbel 9y ago
        >'come at you with a needle and you said, "uh, I didn't agree to that,"' Keep in mind this would happen after you are already drugged up. I don't want to say the compound but it is one which would be thought to make you more agreeable to such things (and from my experience definitely works).
  - BEEdwards 9y ago
    Even when talking about the most egregious breaches throwing out the data doesn't unviolate any patients rights, in fact it makes the violation of their rights worse in my opinion as now no good will come out of it.
    - ska 9y ago
      You have to balance this against future abuses of the system. The value that comes out of it might be better protocols across many other studies.
    - xenophonf 9y ago
      Protocol violations invalidate study data by definition. Cherry picking the data would just bias it worse.
  - kem 9y ago
    It's hard to tell from either article what's going on. The Science piece you linked provides a bit more detail, but I wouldn't say those details are particularly damning (about half of the audited healthy participants didn't have an approved physical as part of the study, but they did have physicals elsewhere at the institution because they were participating in other studies. The most concerning thing is that 2 women didn't have pregnancy test results confirming they weren't pregnant. Sure, you have to be careful about scientists maintaining ethical practice, but the other side of this is a trend toward research IRBs being focused on legal technicalities in reviewing research. For the most part, whether you're talking about NIH or universities, IRBs tend to focus on legal liability rather than ethics. This sounds like it might be reasonable, but in practice what it means is that IRBs aren't asking "is something really problematic happening here?" but instead "is there anything, even the smallest irrelevant detail, that could be used against us in a legal case?" Usually this means that IRBs are representing the institutions' best interests, rather than those of either the researchers or participants. A related thing came up with social scientists recently, where a bunch of them apparently said "screw this, we've had enough"(https://www.nytimes.com/2017/05/22/science/social-science-research-institutional-review-boards-common-rule.html https://www.nytimes.com/2017/05/22/science/social-science-re...). The issue wasn't that they don't care about ethics, it's that they are fed up with IRBs treating the risks of asking people about their political attitudes as if they are equal to having a surgical procedure done. It sounds like there was some drift from protocol, which should be scrutinized. But nothing I've read so far suggests that the harm incurred justifies a ban on the use of the research data already collected. Maybe have some external review committee review the situation and make recommendations for changes; maybe indefinitely postpone future data collection by certain personnel, until things can be ironed out. But to ban publication of anything based on existing data? To me this story illustrates two problems with academic research today: the unreasonableness of IRBs, and the corruption of grant politics.
deleted 9y ago
[deleted]
Shivetya 9y ago
So which group has the overall authority to decide one way or another is the data is valid and if the restrictions are warranted? Who is the parent? NIH? Someone else? Seems odd that a professional level pissing match could get so out of control? is this common?
mnutsch 9y ago
The most important footnote in this article: "Returns vary significantly by school and major."
- larrik 9y ago
  I think you posted to the wrong article.
nonbel 9y ago
Once again, just go back to the old school ways. These new ways are not robust at all. You need to regularly perform independent replications (no, it shouldn't be done in parallel and published in the same paper). This will filter fraud, honest errors, and non-stationary effects. It will also encourage publication of usable methods sections, and much more.
- nonbel 9y ago
  I know we aren't supposed to comment on the votes, but it is notable that the scientific method gets downvoted here...

79 points by endswapper 9y ago

Whenever I read stories like these I tend to take it with a grain of salt. There is always another part of the story we are missing. It will however be interesting to follow the development of this. Hopefully Washington Post or another outlet follows up with this.

xenophonf 9y ago
Here's the backstory: http://www.sciencemag.org/news/2017/02/firing-veteran-nih-scientist-prompts-protests-over-publication-ban http://www.sciencemag.org/news/2017/02/firing-veteran-nih-sc... It sounds like these junior postdocs are trying to salvage their careers after getting seriously screwed over by Braun's misfeasance. That sucks but I'm not sure the data management issues are as trivial as they claim. Experimental protocols are designed to protect the safety of human subjects. Those protections exist because of past transgressions that lead to serious harm and suffering---remember the Tuskegee syphilis experiment? NIDCD is probably right to throw out the data if the protocol was violated for nearly half the study participants covered by their audit.
- nonbel 9y ago
  True story, I once went to try out being a subject for one of these take a drug then do some tests with an EEG hooked up, etc kind of study. They never once mentioned it, but then in the informed consent form there was a whole section agreeing to a spinal tap. I was like wtf, and they said "sorry we gave you an old version on accident" then replaced the form with one without the spinal tap section. The study seemed legit so it probably was just an honest error, but I wonder how many people agreed to a spinal tap without realizing what it was.
  - Cass 9y ago
    Research/academia can certainly be kind of screwed up, but I doubt that any ethics board in a developed country is going to okay a study that involves tricking people into a spinal tap. And all the consent forms patients are asked to sign before agreeing to a procedure, the ones that tell you about the procedure and the risks in excruciating detail, are as much for the protection of the doctor as the patient. That way, if the patient ends up paralyzed, at least they can't claim you didn't warn them when the case goes to court. So unless you were dealing with a single rogue Mad Scientist, I'd assume that was an error.
    - nonbel 9y ago
      I dunno, csf samples (and corresponding data) can be quite valuable. For example, I know someone who had a spinal cyst removed and learned that to this day they do not understand if the fluid is coming from inside or outside the cyst. We lack basic mechanistic information about these issues because there is so little data (then again the papers I looked at that did have such data only checked averages and otherwise weak/incorrect analysis).
  - kevin_thibedeau 9y ago
    Maybe it was a sugar pill and they were just screening for who would read the fine print.
  - arkades 9y ago
    There isn't really any trickery here. If at some point they were to come at you with a needle and you said, "uh, I didn't agree to that," that's where it would end. Consent forms for treatment are only valid until you withdraw consent.
    - nonbel 9y ago
      >'come at you with a needle and you said, "uh, I didn't agree to that,"' Keep in mind this would happen after you are already drugged up. I don't want to say the compound but it is one which would be thought to make you more agreeable to such things (and from my experience definitely works).
- BEEdwards 9y ago
  Even when talking about the most egregious breaches throwing out the data doesn't unviolate any patients rights, in fact it makes the violation of their rights worse in my opinion as now no good will come out of it.
  - ska 9y ago
    You have to balance this against future abuses of the system. The value that comes out of it might be better protocols across many other studies.
  - xenophonf 9y ago
    Protocol violations invalidate study data by definition. Cherry picking the data would just bias it worse.
- kem 9y ago
  It's hard to tell from either article what's going on. The Science piece you linked provides a bit more detail, but I wouldn't say those details are particularly damning (about half of the audited healthy participants didn't have an approved physical as part of the study, but they did have physicals elsewhere at the institution because they were participating in other studies. The most concerning thing is that 2 women didn't have pregnancy test results confirming they weren't pregnant. Sure, you have to be careful about scientists maintaining ethical practice, but the other side of this is a trend toward research IRBs being focused on legal technicalities in reviewing research. For the most part, whether you're talking about NIH or universities, IRBs tend to focus on legal liability rather than ethics. This sounds like it might be reasonable, but in practice what it means is that IRBs aren't asking "is something really problematic happening here?" but instead "is there anything, even the smallest irrelevant detail, that could be used against us in a legal case?" Usually this means that IRBs are representing the institutions' best interests, rather than those of either the researchers or participants. A related thing came up with social scientists recently, where a bunch of them apparently said "screw this, we've had enough"(https://www.nytimes.com/2017/05/22/science/social-science-research-institutional-review-boards-common-rule.html https://www.nytimes.com/2017/05/22/science/social-science-re...). The issue wasn't that they don't care about ethics, it's that they are fed up with IRBs treating the risks of asking people about their political attitudes as if they are equal to having a surgical procedure done. It sounds like there was some drift from protocol, which should be scrutinized. But nothing I've read so far suggests that the harm incurred justifies a ban on the use of the research data already collected. Maybe have some external review committee review the situation and make recommendations for changes; maybe indefinitely postpone future data collection by certain personnel, until things can be ironed out. But to ban publication of anything based on existing data? To me this story illustrates two problems with academic research today: the unreasonableness of IRBs, and the corruption of grant politics.

deleted 9y ago

[deleted]

Shivetya 9y ago

So which group has the overall authority to decide one way or another is the data is valid and if the restrictions are warranted? Who is the parent? NIH? Someone else? Seems odd that a professional level pissing match could get so out of control? is this common?

mnutsch 9y ago

The most important footnote in this article: "Returns vary significantly by school and major."

larrik 9y ago
I think you posted to the wrong article.

nonbel 9y ago

Once again, just go back to the old school ways. These new ways are not robust at all. You need to regularly perform independent replications (no, it shouldn't be done in parallel and published in the same paper). This will filter fraud, honest errors, and non-stationary effects. It will also encourage publication of usable methods sections, and much more.

nonbel 9y ago
I know we aren't supposed to comment on the votes, but it is notable that the scientific method gets downvoted here...