6 ms·

While I agree that the author is being a little facetious (with the constant Nazi references and such), I don't think he is arguing in favor of removing ethical

While I agree that the author is being a little facetious (with the constant Nazi references and such), I don't think he is arguing in favor of removing ethical considerations in research, or even arbitrarily weakening the constraints put on scientific studies. But what purpose is having a review board made up of people who can think for themselves if their behavior is going to be indistinguishable from a program running through a checklist without any care whatsoever for whether or not the checklist makes sense in that context? Massive inefficiencies in bureaucracy are always hidden behind the fig leaf of consistency and accountability, but it's worthwhile to consider what aim they serve and at what cost.

ak_yo 9y ago
I agree that IRBs are not great -- the book cited above does a great job of unpacking how IRBs fail to live up to their promises, and this covers some good additional examples. I also agree that the author isn't necessarily advocating against ethical review as such. I'm more concerned with the author's insistence that he knew the right way to conduct his study ethically. Peer review is a core scientific principle-- we don't allow researchers to make autonomous judgments about the quality of their work.
- justabystander 9y ago
  I don't think he's advocating that he should be setting the rules or self-evaluating. He just wants the rules to be sensible and sensibly enforced. He has justification to complain if the modifications they requested violated safety policies, risked patient well-being or defeated the purpose of the study. Policy that doesn't add any benefit deserves to be questioned. Not to mention that the questions were already being asked. He just wanted to ask them earlier to compare against the eventual diagnosis. It's not the kind of study that should have been abandoned in frustration after two years, as happened here.
  - chris_wot 9y ago
    I think for his next study he should do one into the mental health of those who interact with Institutional Review Boards. If nothing else, it would have a nice recursive effect on the authors.
  - vacri 9y ago
    So, an example of how the author doesn't understand the point of these requirements is the 'encryption' (anonymisation) process, added to 'the files had to be kept together!'. The anonymisation process isn't about stolen files. It's about data leakage, and keeping identifying data out of result sets. If you're pulling your all-nighter to prep for the big talk tomorrow, for example, you're not going to accidentally include identifying data from your results page if it doesn't exist on your results page in the first place. Edit: more important for the scientific process (rather than privacy) than 'data leakage' is 'data sharing', as yread points out below. Anonymised data can be quickly and safely shared, and others can run their analyses on your results. Non-anonymised data can't be shared. If you're interested in publishing a robust scientific paper, why would you be against opening your data for inspection? > Not to mention that the questions were already being asked. He just wanted to ask them earlier to compare against the eventual diagnosis. It's not the kind of study that should have been abandoned in frustration after two years, as happened here. Then he could have just asked them and done his own informal study. Nothing is stopping the doctor from saying "are you happy, then sad?" on first meeting a patient. But if you want to do a formal, publishable study, then you should have all your ducks in a row. Make sure your independent variables are properly controlled, make sure any ethical issues have been externally vetted, so on and so forth. While the IRB certainly had some annoying concerns, so much of this author's frustration just simply wouldn't be there if he understood why those questions were being asked.
    - rocqua 9y ago
      > make sure any ethical issues have been externally vetted, so on and so forth. Sure, but the complaints of the IRB mentioned (and the auditor) seem to be far beyond ensuring practice is ethical. Instead, they seem to focus on following process only for the sake of process. Why should the consent form have the title of the study? Why should the consent form contain a list of risks when there are none? What is wrong with having consent forms signed with pencil when pens aren't allowed. Why should the data integrity plan require periodic review (i.e. why should we have a data integrity plan - integrity plan). These are all indicative of a bureaucratic system that places too much emphasis on 'process', losing sight of 'outcome' in the end.
  - csydas 9y ago
    I think it's quite easy to see the benefit of the author's study and their frustration is completely understandable - however, I can't help but feel that much of their frustration was simply because the rules impeded their progress, not that the rules were actually useless. My major gripe whenever there is a long piece that decries the bureaucracy of various regulatory boards is that the complaints tend to be about how the bureaucracy is a personal inconvenience. Some of the gripes I'll absolutely grant; pen versus pencil and inflexibility for giving potentially violent persons a weapon probably needs some sort of leeway, but I think protectionary measures absolutely should be a brick wall; a surmountable one, sure, but only as a result of you actually trying a bit and demonstrating that your intended actions aren't going to do exactly what the regulation is trying to prevent, and that should be on the researcher using human participants to demonstrate. The idea of easily avoidable and mutable regulatory functions seems contradictory - that is, a researcher shouldn't be declaring what should and should not apply to them. This isn't fear that they're Hitler and going to inject people with nastiness, it's fear of the dumb mistakes that every human makes and our often poor ability to predict the outcome of certain actions. I get it - they want to help people and the regulations are inconvenient for them; but having gone through many IRB processes myself, it's not insurmountable in the least bit, much less anything the author listed.
icelancer 9y ago
>>Massive inefficiencies in bureaucracy are always hidden behind the fig leaf of consistency and accountability, but it's worthwhile to consider what aim they serve and at what cost. This problem was largely solved by the development and acceptance of private IRBs. Going with private IRBs that meet 2x weekly, turn forms around quickly, and are willing to pre-qualify your work - at cost, of course - is the road many researchers take now.
- Sir_Substance 9y ago
  How is that solving the problem???
  - icelancer 9y ago
    ....how does it not? If you are asking the question: "How does this private market solution solve the problem of government/education/medical bureaucracy being inefficient," then I guess it does not solve the root cause. But trying to solve IRB problems involving that three-headed hydra is not likely to succeed. Going around the system is a feasible workaround.
    - jessriedel 9y ago
      Implementing a terrible process faster at great cost is not the same thing as solving the problem, nor is it the best possible solution short of "fix all bureaucracy".
      - deleted 9y ago
        [deleted]
      - icelancer 9y ago
        I wouldn't call my experiences in private IRB to be terrible at all, honestly.
        jessriedel 9y ago
Balgair 9y ago
> But what purpose is having a review board made up of people who can think for themselves if their behavior is going to be indistinguishable from a program running through a checklist without any care whatsoever for whether or not the checklist makes sense in that context? In general, most IRBs are there to help out, not to get in the way. It may feel like getting in the way, at the time, but they mostly are there to help. For example: the wordings of your consent forms. FDA says that they need to be at an 8th grade level, +/- 0.2 grade levels. You can game MS word to get that to an 8.2 grade level but have it be 140+ pages, and a lot of researchers do that. This is especially prevalent in clinical drug trials where many Pharam companies try to make the consent form a contract filled with legalese. That egregious example is non typical, but highlights the issues with consent forms being readable and an actual consent form a person. This is where a decent IRB would help out and tell you that you need to re-write the consent form to be an something that is not gibberish. (Side-note: Most newspapers are written at a 6th grade level it turns out). Another example is International research. There was a case I heard of where a group was going to go collect HIV data in Tanzania. They were just going to go out and collect blood samples and sexual health questions alongside. The part of Tanzania they were collecting from is rural and believes in very strange medical practices like eating albino people or having sex with virgins to cure HIV, though I forget exactly what. Balancing the local beliefs with data collection can be tricky in that case. Typically, you would like a local IRB in that region, but there was none. So the US based IRB decided that the researchers needed to go out to the religious and government centers in that part of Tanzania and ask them what to do and what questions they could ask, etc. This added a lot of time and expenses to the study to spend in country before data collection, but the US IRB held firm on that. This was a good idea, as the local peoples had a lot of reservations and wanted to see all the data themselves before it was sent out of country, so as to go off and then kill women that were HIV positive. If the US researchers had found this out beforehand, a lot of deaths may have occurred. They subsequently then left off a lot of questions and data gathering. Researchers are human too and we forget things all the time and have a lot of pressures on us as well. IRBs should be there to help out and not be a road-block. If the IRB you are working with does turn out to be that, talk to other researchers and find if they have the same issues. If you all do, go to the IRB and explain to them that they are unreasonable, most of them do want to know that they are holding things up unnecessarily. If it persists, go to the hospital/university admin as a group and make the case. These advancements need to be triple checked, but they still need to go through to begin with. IRBs are not evil, sclerotic maybe, but not evil. Work hard to get them to be better, we are all in this together.

ak_yo 9y ago

I agree that IRBs are not great -- the book cited above does a great job of unpacking how IRBs fail to live up to their promises, and this covers some good additional examples. I also agree that the author isn't necessarily advocating against ethical review as such. I'm more concerned with the author's insistence that he knew the right way to conduct his study ethically. Peer review is a core scientific principle-- we don't allow researchers to make autonomous judgments about the quality of their work.

justabystander 9y ago
I don't think he's advocating that he should be setting the rules or self-evaluating. He just wants the rules to be sensible and sensibly enforced. He has justification to complain if the modifications they requested violated safety policies, risked patient well-being or defeated the purpose of the study. Policy that doesn't add any benefit deserves to be questioned. Not to mention that the questions were already being asked. He just wanted to ask them earlier to compare against the eventual diagnosis. It's not the kind of study that should have been abandoned in frustration after two years, as happened here.
- chris_wot 9y ago
  I think for his next study he should do one into the mental health of those who interact with Institutional Review Boards. If nothing else, it would have a nice recursive effect on the authors.
- vacri 9y ago
  So, an example of how the author doesn't understand the point of these requirements is the 'encryption' (anonymisation) process, added to 'the files had to be kept together!'. The anonymisation process isn't about stolen files. It's about data leakage, and keeping identifying data out of result sets. If you're pulling your all-nighter to prep for the big talk tomorrow, for example, you're not going to accidentally include identifying data from your results page if it doesn't exist on your results page in the first place. Edit: more important for the scientific process (rather than privacy) than 'data leakage' is 'data sharing', as yread points out below. Anonymised data can be quickly and safely shared, and others can run their analyses on your results. Non-anonymised data can't be shared. If you're interested in publishing a robust scientific paper, why would you be against opening your data for inspection? > Not to mention that the questions were already being asked. He just wanted to ask them earlier to compare against the eventual diagnosis. It's not the kind of study that should have been abandoned in frustration after two years, as happened here. Then he could have just asked them and done his own informal study. Nothing is stopping the doctor from saying "are you happy, then sad?" on first meeting a patient. But if you want to do a formal, publishable study, then you should have all your ducks in a row. Make sure your independent variables are properly controlled, make sure any ethical issues have been externally vetted, so on and so forth. While the IRB certainly had some annoying concerns, so much of this author's frustration just simply wouldn't be there if he understood why those questions were being asked.
  - rocqua 9y ago
    > make sure any ethical issues have been externally vetted, so on and so forth. Sure, but the complaints of the IRB mentioned (and the auditor) seem to be far beyond ensuring practice is ethical. Instead, they seem to focus on following process only for the sake of process. Why should the consent form have the title of the study? Why should the consent form contain a list of risks when there are none? What is wrong with having consent forms signed with pencil when pens aren't allowed. Why should the data integrity plan require periodic review (i.e. why should we have a data integrity plan - integrity plan). These are all indicative of a bureaucratic system that places too much emphasis on 'process', losing sight of 'outcome' in the end.
- csydas 9y ago
  I think it's quite easy to see the benefit of the author's study and their frustration is completely understandable - however, I can't help but feel that much of their frustration was simply because the rules impeded their progress, not that the rules were actually useless. My major gripe whenever there is a long piece that decries the bureaucracy of various regulatory boards is that the complaints tend to be about how the bureaucracy is a personal inconvenience. Some of the gripes I'll absolutely grant; pen versus pencil and inflexibility for giving potentially violent persons a weapon probably needs some sort of leeway, but I think protectionary measures absolutely should be a brick wall; a surmountable one, sure, but only as a result of you actually trying a bit and demonstrating that your intended actions aren't going to do exactly what the regulation is trying to prevent, and that should be on the researcher using human participants to demonstrate. The idea of easily avoidable and mutable regulatory functions seems contradictory - that is, a researcher shouldn't be declaring what should and should not apply to them. This isn't fear that they're Hitler and going to inject people with nastiness, it's fear of the dumb mistakes that every human makes and our often poor ability to predict the outcome of certain actions. I get it - they want to help people and the regulations are inconvenient for them; but having gone through many IRB processes myself, it's not insurmountable in the least bit, much less anything the author listed.

icelancer 9y ago

>>Massive inefficiencies in bureaucracy are always hidden behind the fig leaf of consistency and accountability, but it's worthwhile to consider what aim they serve and at what cost. This problem was largely solved by the development and acceptance of private IRBs. Going with private IRBs that meet 2x weekly, turn forms around quickly, and are willing to pre-qualify your work - at cost, of course - is the road many researchers take now.

Sir_Substance 9y ago
How is that solving the problem???
- icelancer 9y ago
  ....how does it not? If you are asking the question: "How does this private market solution solve the problem of government/education/medical bureaucracy being inefficient," then I guess it does not solve the root cause. But trying to solve IRB problems involving that three-headed hydra is not likely to succeed. Going around the system is a feasible workaround.
  - jessriedel 9y ago
    Implementing a terrible process faster at great cost is not the same thing as solving the problem, nor is it the best possible solution short of "fix all bureaucracy".
    - deleted 9y ago
      [deleted]
    - icelancer 9y ago
      I wouldn't call my experiences in private IRB to be terrible at all, honestly.
      - jessriedel 9y ago

Balgair 9y ago

> But what purpose is having a review board made up of people who can think for themselves if their behavior is going to be indistinguishable from a program running through a checklist without any care whatsoever for whether or not the checklist makes sense in that context? In general, most IRBs are there to help out, not to get in the way. It may feel like getting in the way, at the time, but they mostly are there to help. For example: the wordings of your consent forms. FDA says that they need to be at an 8th grade level, +/- 0.2 grade levels. You can game MS word to get that to an 8.2 grade level but have it be 140+ pages, and a lot of researchers do that. This is especially prevalent in clinical drug trials where many Pharam companies try to make the consent form a contract filled with legalese. That egregious example is non typical, but highlights the issues with consent forms being readable and an actual consent form a person. This is where a decent IRB would help out and tell you that you need to re-write the consent form to be an something that is not gibberish. (Side-note: Most newspapers are written at a 6th grade level it turns out). Another example is International research. There was a case I heard of where a group was going to go collect HIV data in Tanzania. They were just going to go out and collect blood samples and sexual health questions alongside. The part of Tanzania they were collecting from is rural and believes in very strange medical practices like eating albino people or having sex with virgins to cure HIV, though I forget exactly what. Balancing the local beliefs with data collection can be tricky in that case. Typically, you would like a local IRB in that region, but there was none. So the US based IRB decided that the researchers needed to go out to the religious and government centers in that part of Tanzania and ask them what to do and what questions they could ask, etc. This added a lot of time and expenses to the study to spend in country before data collection, but the US IRB held firm on that. This was a good idea, as the local peoples had a lot of reservations and wanted to see all the data themselves before it was sent out of country, so as to go off and then kill women that were HIV positive. If the US researchers had found this out beforehand, a lot of deaths may have occurred. They subsequently then left off a lot of questions and data gathering. Researchers are human too and we forget things all the time and have a lot of pressures on us as well. IRBs should be there to help out and not be a road-block. If the IRB you are working with does turn out to be that, talk to other researchers and find if they have the same issues. If you all do, go to the IRB and explain to them that they are unreasonable, most of them do want to know that they are holding things up unnecessarily. If it persists, go to the hospital/university admin as a group and make the case. These advancements need to be triple checked, but they still need to go through to begin with. IRBs are not evil, sclerotic maybe, but not evil. Work hard to get them to be better, we are all in this together.