5 ms·

For those who think the author is in the wrong 1 - Do you think this story indicates a success or a failure of the IRB? 2 - Would you rather a better expedite

For those who think the author is in the wrong 1 - Do you think this story indicates a success or a failure of the IRB? 2 - Would you rather a better expedited review process AND more studies (remember the personal effort cost killed this study), neither, or do you see some 3rd option?

icelancer 9y ago
>>Do you think this story indicates a success or a failure of the IRB? Failure of the IRB but mostly failure of the author to explore his options. >>Would you rather a better expedited review process AND more studies (remember the personal effort cost killed this study), neither, or do you see some 3rd option? As I've posted in this thread, this problem is solved. Private IRBs exist that are much more responsive.
- tedunangst 9y ago
  What options do you think were inadequately explored to sign the consent forms? Finger paint?
  - icelancer 9y ago
    >>What options do you think were inadequately explored to sign the consent forms? None. Now, if you asked a question without assuming what you think I was saying, I'd tell you that Alexander's other options were to not use the IRB that he went to, and to explore the private marketplace of IRB organizations that provide much better service than the nightmare he dealt with - a very common practice in all fields.
    - mattkrause 9y ago
      > Alexander's other options were to not use the IRB that he went to This is, frankly, ridiculous. The 50-70% overhead the university takes from every grant already pays for one IRB. If that one isn't working, the solution is not to tap in (almost inevitably tight) research funds to pay for a second for-profit one instead. Given how hard it is to charge things like computers to grants, I'm actually amazed this is even allowed.
      - icelancer 9y ago
        >> If that one isn't working, the solution is not to tap in (almost inevitably tight) research funds to pay for a second for-profit one instead. I am curious: How much do you think private IRB engagement costs?
        euyyn 9y ago
        I'm not the parent but would like to know. How much does it cost? And how does "I skipped our free IRB and paid this third-party IRB because they're laxer" really look to the department?
        icelancer 9y ago
        A few hundred dollars, generally. Depending on provider. >>And how does "I skipped our free IRB and paid this third-party IRB because they're laxer" really look to the department? In my experience the third-party IRB orgs are not more lax, they are just faster at turning things around. Alexander's article bemoans primarily the time it took to get all these decisions made, which private IRBs would be on top of. I got my rejections of consent forms within 24 hours and get my docs pre-qualified over the phone after submission and discussion with reps. Try that at your local university's IRB. Alexander also has complaints about the questions and methods, but they're mostly unfounded. The study title aspect is important, the signing in pencil is not really relevant (I get his complaint but most industries don't allow contracts signed in pencil), and the transfer of the study to a new investigator unannounced is a huge breach of procedure and he's lucky that all the IRB asked was for a submission of a new investigator. Usually it is much more arduous. IRB simply controls the ethics and the informed consent documents and has the right to audit materials and process at any time to ensure you stay up on these things. Beyond that, it doesn't do much. When you submit your paper to a peer-reviewed journal (some who don't even need IRB approval, just an ethics statement, mind you), they don't call your IRB provider and reference check you. Ohio State researchers published a fraudulent article about CrossFit years ago in the JSCR, which is considered a very prestigious exercise science journal. http://retractionwatch.com/2017/06/02/journal-retracts-ohio-state-crossfit-study-center-lawsuits/ http://retractionwatch.com/2017/06/02/journal-retracts-ohio-... Devor stated they had IRB approval. In actuality, they did not. CrossFit sued NSCA (parent org of JSCR) and won lawyer's fees, and Ohio State settled with CrossFit for six figures for damages due to false publications. IRB is a mere formality in today's landscape. You just need to get it out of the way. It's not like journals are cross-checking your IRB applications, they just want to see the approval that you put in there. And hell, if you're like Ohio State researchers (and probably many more), you can just lie about it.
    - PeterisP 9y ago
      If one group of IRBs is prohibiting certain practices and another is allowing them, then that is obviously a major systematic flaw in IRBs as such. Either one IRB is letting shady practices through (and that's a big problem) or another class of IRBs is unreasonably restricting research, and that's a problem as well. Shopping for a better IRB isn't a systematic solution, it's a workaround for a system of flawed IRBs.
  - PeterisP 9y ago
    It is worthwhile to consider that evidence of consent is mandatory, and signing in pencil is not considered a legally valid option. If you can't sign the standard form for safety reasons, there are all kinds of ways that have been traditionally used to document legal consent for people who can't sign documents for various reasons - for example, public notaries, signatures of witnesses or even simple video evidence of explaining whatever needed to explain and the patient consenting would be a possibility.
euyyn 9y ago
1. It's a success of his IRB. Of the many requirements he described, both by his IRB and the auditors, he didn't make the effort to try to understand how any of them could be good for his subjects. Instead he focused on how they affected his study, and tried to find the bare minimum he had to do to get approved. If the result of it is he doesn't want to deal with IRBs anymore, it might be for the best. 2. I believe this study could have been done with much less personal effort. His hypothesis is the screening question leads to unwarranted diagnosis. He already had Dr. W. convinced that it's plausible, and willing to do full diagnosis to his patients. And he had other doctors that would only use the screening question. All that's required to refute the null hypothesis is compare the rate of "yes" vs "no" in the set of Dr. W's diagnoses, with the rate of the doctors that just use the screening question. If his hypothesis is true, the other doctors would get more positives. That's 1 bit of information he had to extract anonymously from the patients' existing records, which is a strict subset of the methodology he developed for the study.
- skybrian 9y ago
  If you don't test both kinds of diagnostics on the same patients, that doesn't sound the same at all? As another example, if you have a ten-question survey, asking 1000 people one question is not the same as asking 100 people all ten questions. You don't learn nearly as much about the correlation between answers to different questions.
  - euyyn 9y ago
    Assume the patients are assigned to the doctors in a way that's not affected by whether they have the condition or not (e.g. at random). If the screening question doesn't lead to over-diagnosis, then you'd expect the same ratio of positives/negatives from both sets. You need more subjects to attain the same statistical power, but the data is much easier to obtain. 100 subjects already sounded like overkill anyway.
    - YawningAngel 9y ago
      It doesn't strictly pertain to his hypothesis, but presumably you would like data on false positives and negatives as well as overall outcome percentages.

icelancer 9y ago

>>Do you think this story indicates a success or a failure of the IRB? Failure of the IRB but mostly failure of the author to explore his options. >>Would you rather a better expedited review process AND more studies (remember the personal effort cost killed this study), neither, or do you see some 3rd option? As I've posted in this thread, this problem is solved. Private IRBs exist that are much more responsive.

tedunangst 9y ago
What options do you think were inadequately explored to sign the consent forms? Finger paint?
- icelancer 9y ago
  >>What options do you think were inadequately explored to sign the consent forms? None. Now, if you asked a question without assuming what you think I was saying, I'd tell you that Alexander's other options were to not use the IRB that he went to, and to explore the private marketplace of IRB organizations that provide much better service than the nightmare he dealt with - a very common practice in all fields.
  - mattkrause 9y ago
    > Alexander's other options were to not use the IRB that he went to This is, frankly, ridiculous. The 50-70% overhead the university takes from every grant already pays for one IRB. If that one isn't working, the solution is not to tap in (almost inevitably tight) research funds to pay for a second for-profit one instead. Given how hard it is to charge things like computers to grants, I'm actually amazed this is even allowed.
    - icelancer 9y ago
      >> If that one isn't working, the solution is not to tap in (almost inevitably tight) research funds to pay for a second for-profit one instead. I am curious: How much do you think private IRB engagement costs?
      - euyyn 9y ago
        I'm not the parent but would like to know. How much does it cost? And how does "I skipped our free IRB and paid this third-party IRB because they're laxer" really look to the department?
        icelancer 9y ago
        A few hundred dollars, generally. Depending on provider. >>And how does "I skipped our free IRB and paid this third-party IRB because they're laxer" really look to the department? In my experience the third-party IRB orgs are not more lax, they are just faster at turning things around. Alexander's article bemoans primarily the time it took to get all these decisions made, which private IRBs would be on top of. I got my rejections of consent forms within 24 hours and get my docs pre-qualified over the phone after submission and discussion with reps. Try that at your local university's IRB. Alexander also has complaints about the questions and methods, but they're mostly unfounded. The study title aspect is important, the signing in pencil is not really relevant (I get his complaint but most industries don't allow contracts signed in pencil), and the transfer of the study to a new investigator unannounced is a huge breach of procedure and he's lucky that all the IRB asked was for a submission of a new investigator. Usually it is much more arduous. IRB simply controls the ethics and the informed consent documents and has the right to audit materials and process at any time to ensure you stay up on these things. Beyond that, it doesn't do much. When you submit your paper to a peer-reviewed journal (some who don't even need IRB approval, just an ethics statement, mind you), they don't call your IRB provider and reference check you. Ohio State researchers published a fraudulent article about CrossFit years ago in the JSCR, which is considered a very prestigious exercise science journal. http://retractionwatch.com/2017/06/02/journal-retracts-ohio-state-crossfit-study-center-lawsuits/ http://retractionwatch.com/2017/06/02/journal-retracts-ohio-... Devor stated they had IRB approval. In actuality, they did not. CrossFit sued NSCA (parent org of JSCR) and won lawyer's fees, and Ohio State settled with CrossFit for six figures for damages due to false publications. IRB is a mere formality in today's landscape. You just need to get it out of the way. It's not like journals are cross-checking your IRB applications, they just want to see the approval that you put in there. And hell, if you're like Ohio State researchers (and probably many more), you can just lie about it.
  - PeterisP 9y ago
    If one group of IRBs is prohibiting certain practices and another is allowing them, then that is obviously a major systematic flaw in IRBs as such. Either one IRB is letting shady practices through (and that's a big problem) or another class of IRBs is unreasonably restricting research, and that's a problem as well. Shopping for a better IRB isn't a systematic solution, it's a workaround for a system of flawed IRBs.
- PeterisP 9y ago
  It is worthwhile to consider that evidence of consent is mandatory, and signing in pencil is not considered a legally valid option. If you can't sign the standard form for safety reasons, there are all kinds of ways that have been traditionally used to document legal consent for people who can't sign documents for various reasons - for example, public notaries, signatures of witnesses or even simple video evidence of explaining whatever needed to explain and the patient consenting would be a possibility.

euyyn 9y ago

1. It's a success of his IRB. Of the many requirements he described, both by his IRB and the auditors, he didn't make the effort to try to understand how any of them could be good for his subjects. Instead he focused on how they affected his study, and tried to find the bare minimum he had to do to get approved. If the result of it is he doesn't want to deal with IRBs anymore, it might be for the best. 2. I believe this study could have been done with much less personal effort. His hypothesis is the screening question leads to unwarranted diagnosis. He already had Dr. W. convinced that it's plausible, and willing to do full diagnosis to his patients. And he had other doctors that would only use the screening question. All that's required to refute the null hypothesis is compare the rate of "yes" vs "no" in the set of Dr. W's diagnoses, with the rate of the doctors that just use the screening question. If his hypothesis is true, the other doctors would get more positives. That's 1 bit of information he had to extract anonymously from the patients' existing records, which is a strict subset of the methodology he developed for the study.

skybrian 9y ago
If you don't test both kinds of diagnostics on the same patients, that doesn't sound the same at all? As another example, if you have a ten-question survey, asking 1000 people one question is not the same as asking 100 people all ten questions. You don't learn nearly as much about the correlation between answers to different questions.
- euyyn 9y ago
  Assume the patients are assigned to the doctors in a way that's not affected by whether they have the condition or not (e.g. at random). If the screening question doesn't lead to over-diagnosis, then you'd expect the same ratio of positives/negatives from both sets. You need more subjects to attain the same statistical power, but the data is much easier to obtain. 100 subjects already sounded like overkill anyway.
  - YawningAngel 9y ago
    It doesn't strictly pertain to his hypothesis, but presumably you would like data on false positives and negatives as well as overall outcome percentages.