5 ms·

Another horrific failure of reviewers. 6 subjects!? Did anyone actually check the study they registered? https://clinicaltrials.gov/ct2/show/NCT02790541 https:

Another horrific failure of reviewers. 6 subjects!? Did anyone actually check the study they registered? https://clinicaltrials.gov/ct2/show/NCT02790541 https://clinicaltrials.gov/ct2/show/NCT02790541 Their actual clinical trial had 62 subjects and it had a control arm. The paper has only 6 subjects and no controls. If I run an experiment with 60 people, I can find evidence for anything (precognition, telekinesis, etc.) if I'm allowed to cherry pick 6 data points and drop any controls. The authors should be ashamed for polluting the scientific record with this junk.

parliowl 5y ago
it sounds like you are assuming they collected more data than they reported, is there evidence of that? They preregistered to collect 60, but they couldve simply been unable to collect that many (eg COVID). If it was preregistered perhaps the journal were obligated to publish the results no matter the outcome. Some journals do this ostensibly to combat publication bias. I didn’t read the whole thing: are the authors actually claiming the human data are informative? If so, that is wrong. But if they are just saying “hey, these are the data!” I wouldn’t be so harsh. Edit: Just read the methods. Data collection was 2016 to 2020, so COVID probably wasn’t the issue. They were selected for poor memory yet normal cognition in other domains, so it is possible they couldn’t recruit the number of subjects they proposed who met this criteria and also were willing and able to complete the treatment. I’m now quite confident the memory result is regression to the mean, and therefore nothing to do with the treatment. I also read the portion of the discussion related to cognition. They are claiming to have found something meaningful (in the human data) when they definitely haven’t.
- stuartbman 5y ago
  I would absolutely be harsh on this. There might well be reasons why they didn't include the other 54- perhaps these were the only 6 to maintain cognitive function sufficiently to continue to receive the treatment, or even be alive! This is why we analyse on an intention-to-treat basis rather than who actually received the treatment. The publication makes no such attempt to explain this potential source of bias.
- BeetleB 5y ago
  What, in your mind, is the purpose of preregistration?
- light_hue_1 5y ago
  > I wouldn’t be so harsh. I'm not being harsh. This is basic due diligence that the editors and reviewers at Aging must do, by their own rules. Aging agreed to follow the ICMJE (International Committee of Medical Journal Editors) rules (for many reasons including pressure from funding agencies). Those rules are fairly extensive. Among them is that you register your trial, then after you're done you must complete your trial and update it. The ICMJE rules literally say "The purpose of clinical trial registration is to prevent selective publication and selective reporting of research outcomes" The 62 patients aren't theoretical. They actually existed and were enrolled in the trial. That's why the preregistration says > Actual Enrollment : 62 participants Before the authors completed the trial it would have said "Anticipated Enrollment". They had to update this field to be able to publish according to the ICMJE. So this data exists. What happened to it? Did the authors cherry pick? Then they could show that God exists. Did the authors exclude subjects because they died, declined too much, developed diabetes, etc? Then that completely biases their results. They need to carefully explain what happened. They need to correct for this source of bias in all of their statistics and can't just publish whatever they want and pretend the study as registered didn't exist. This wasn't a mistake either. If you look at the data sharing section they said they're undecided if they will give you the raw data to check their results. I wonder why? But it gets worse. This violates a whole list of Aging's rules. https://www.aging-us.com/for-authors https://www.aging-us.com/for-authors > Aging supports the position of the International Committee of Medical Journal Editors (ICMJE) on trial registration. All trials initiated after 1 July 2005 must be registered prospectively in a publicly accessible registry (i.e., before patient recruitment has begun), or they will not be considered for publication. Authors of randomized controlled trials must adhere to the CONSORT reporting guidelines appropriate to their trial design. Please check the CONSORT statement Web site for information on the appropriate guidelines for specific trial types. Before the paper can enter peer review authors must: 1) name in the paper trial registry, trial registration number, and IRB and 2) provide a copy of the trial protocol and a completed CONSORT checklist as supporting files. The CONSORT flow diagram must be included as Figure 1. Any deviation from the trial protocol must be explained in the paper. Authors must explicitly discuss informed consent in their paper, and AGING reserves the right to ask for a copy of the patient consent form. Information on statistical methods or participants beyond what is indicated in the CONSORT statement should be reported in the Methods section. Where's the CONSORT checklist? Where's the CONSORT flow diagram? "Any deviation from the trial protocol must be explained in the paper" Let's look at another random paper from the same edition of Aging. "Association between tooth loss rate and risk of mild cognitive impairment in older adults: a population-based longitudinal study". https://www.aging-us.com/article/203504/pdf https://www.aging-us.com/article/203504/pdf Figure 1 is the CONSORT flow diagram as required by Aging's own rules. This is a boondoggle that doesn't even meet the most basic criteria for scientific publications.
adamdusty 5y ago
I disagree. Its more just sensationalization by journalists. The actual paper talks more about an existing model for mice and how their findings are consistent with the model. The most outrageous claim they make is that the study demonstrates the efficacy of HBOT on Alzheimers, which is technically the truth but misleading in the sense that it demonstrates how effective it could be. The authors should be happy to have added to ongoing alzheimers research. Their actual claims are much less bold than the article makes them out to be.
- vkou 5y ago
  I thought that mice don't get Alzheimer's, and thereby, this field is littered with mice studies that have zero bearing on humans?
  - something98 5y ago
    This is true (and rarely mentioned); rodents don't get AD. But they can be genetically engineered to produce the proteins that researchers consider the hallmarks of AD. The fact that this is a circular definition never fails to frustrate me when discussing the misuse of animals for AD research. (and the fact that billions have been wasted on proving that this circular definition doesn't apply to humans is especially sad)
- light_hue_1 5y ago
  Oh how I wish that were true! Journalists aren't sensationalizing anything here. Direct quotes from the paper: > Motivated by these findings, we exposed elderly patients with significant memory loss at baseline to HBOT and observed an increase in CBF and improvement in cognitive performances. This study demonstrates HBOT efficacy in hypoxia-related neurological conditions, particularly in AD and aging. > In summary, we showed here that HBOT offers multi-faceted neuroprotective effects on the complex pathology of Alzheimer’s disease and also improves CBF and cognition in humans. > Given that HBOT is considered a safe and tolerable treatment currently being used in the clinic, the increasing number of clinical trials showing that HBOT improves cognitive function in patients suffering from chronic brain damage, the pre-clinical studies elucidating mechanisms of HBOT action, and the fact that there is presently no effective intervention for AD, HBOT should be considered as a therapeutic approach to slow the progression or even improve the pathophysiology responsible for this disease. I can't imagine bolder claims than "we showed here that HBOT ... improves CBF and cognition in humans" and "HBOT should be considered as a therapeutic approach". I mean this literally. I can't imagine it. What else could they have said that would be more direct aside from HBOT will cure you 100% guaranteed?
mirekrusin 5y ago
Maybe 56 died?

parliowl 5y ago

it sounds like you are assuming they collected more data than they reported, is there evidence of that? They preregistered to collect 60, but they couldve simply been unable to collect that many (eg COVID). If it was preregistered perhaps the journal were obligated to publish the results no matter the outcome. Some journals do this ostensibly to combat publication bias. I didn’t read the whole thing: are the authors actually claiming the human data are informative? If so, that is wrong. But if they are just saying “hey, these are the data!” I wouldn’t be so harsh. Edit: Just read the methods. Data collection was 2016 to 2020, so COVID probably wasn’t the issue. They were selected for poor memory yet normal cognition in other domains, so it is possible they couldn’t recruit the number of subjects they proposed who met this criteria and also were willing and able to complete the treatment. I’m now quite confident the memory result is regression to the mean, and therefore nothing to do with the treatment. I also read the portion of the discussion related to cognition. They are claiming to have found something meaningful (in the human data) when they definitely haven’t.

stuartbman 5y ago
I would absolutely be harsh on this. There might well be reasons why they didn't include the other 54- perhaps these were the only 6 to maintain cognitive function sufficiently to continue to receive the treatment, or even be alive! This is why we analyse on an intention-to-treat basis rather than who actually received the treatment. The publication makes no such attempt to explain this potential source of bias.
BeetleB 5y ago
What, in your mind, is the purpose of preregistration?
light_hue_1 5y ago
> I wouldn’t be so harsh. I'm not being harsh. This is basic due diligence that the editors and reviewers at Aging must do, by their own rules. Aging agreed to follow the ICMJE (International Committee of Medical Journal Editors) rules (for many reasons including pressure from funding agencies). Those rules are fairly extensive. Among them is that you register your trial, then after you're done you must complete your trial and update it. The ICMJE rules literally say "The purpose of clinical trial registration is to prevent selective publication and selective reporting of research outcomes" The 62 patients aren't theoretical. They actually existed and were enrolled in the trial. That's why the preregistration says > Actual Enrollment : 62 participants Before the authors completed the trial it would have said "Anticipated Enrollment". They had to update this field to be able to publish according to the ICMJE. So this data exists. What happened to it? Did the authors cherry pick? Then they could show that God exists. Did the authors exclude subjects because they died, declined too much, developed diabetes, etc? Then that completely biases their results. They need to carefully explain what happened. They need to correct for this source of bias in all of their statistics and can't just publish whatever they want and pretend the study as registered didn't exist. This wasn't a mistake either. If you look at the data sharing section they said they're undecided if they will give you the raw data to check their results. I wonder why? But it gets worse. This violates a whole list of Aging's rules. https://www.aging-us.com/for-authors https://www.aging-us.com/for-authors > Aging supports the position of the International Committee of Medical Journal Editors (ICMJE) on trial registration. All trials initiated after 1 July 2005 must be registered prospectively in a publicly accessible registry (i.e., before patient recruitment has begun), or they will not be considered for publication. Authors of randomized controlled trials must adhere to the CONSORT reporting guidelines appropriate to their trial design. Please check the CONSORT statement Web site for information on the appropriate guidelines for specific trial types. Before the paper can enter peer review authors must: 1) name in the paper trial registry, trial registration number, and IRB and 2) provide a copy of the trial protocol and a completed CONSORT checklist as supporting files. The CONSORT flow diagram must be included as Figure 1. Any deviation from the trial protocol must be explained in the paper. Authors must explicitly discuss informed consent in their paper, and AGING reserves the right to ask for a copy of the patient consent form. Information on statistical methods or participants beyond what is indicated in the CONSORT statement should be reported in the Methods section. Where's the CONSORT checklist? Where's the CONSORT flow diagram? "Any deviation from the trial protocol must be explained in the paper" Let's look at another random paper from the same edition of Aging. "Association between tooth loss rate and risk of mild cognitive impairment in older adults: a population-based longitudinal study". https://www.aging-us.com/article/203504/pdf https://www.aging-us.com/article/203504/pdf Figure 1 is the CONSORT flow diagram as required by Aging's own rules. This is a boondoggle that doesn't even meet the most basic criteria for scientific publications.

adamdusty 5y ago

I disagree. Its more just sensationalization by journalists. The actual paper talks more about an existing model for mice and how their findings are consistent with the model. The most outrageous claim they make is that the study demonstrates the efficacy of HBOT on Alzheimers, which is technically the truth but misleading in the sense that it demonstrates how effective it could be. The authors should be happy to have added to ongoing alzheimers research. Their actual claims are much less bold than the article makes them out to be.

vkou 5y ago
I thought that mice don't get Alzheimer's, and thereby, this field is littered with mice studies that have zero bearing on humans?
- something98 5y ago
  This is true (and rarely mentioned); rodents don't get AD. But they can be genetically engineered to produce the proteins that researchers consider the hallmarks of AD. The fact that this is a circular definition never fails to frustrate me when discussing the misuse of animals for AD research. (and the fact that billions have been wasted on proving that this circular definition doesn't apply to humans is especially sad)
light_hue_1 5y ago
Oh how I wish that were true! Journalists aren't sensationalizing anything here. Direct quotes from the paper: > Motivated by these findings, we exposed elderly patients with significant memory loss at baseline to HBOT and observed an increase in CBF and improvement in cognitive performances. This study demonstrates HBOT efficacy in hypoxia-related neurological conditions, particularly in AD and aging. > In summary, we showed here that HBOT offers multi-faceted neuroprotective effects on the complex pathology of Alzheimer’s disease and also improves CBF and cognition in humans. > Given that HBOT is considered a safe and tolerable treatment currently being used in the clinic, the increasing number of clinical trials showing that HBOT improves cognitive function in patients suffering from chronic brain damage, the pre-clinical studies elucidating mechanisms of HBOT action, and the fact that there is presently no effective intervention for AD, HBOT should be considered as a therapeutic approach to slow the progression or even improve the pathophysiology responsible for this disease. I can't imagine bolder claims than "we showed here that HBOT ... improves CBF and cognition in humans" and "HBOT should be considered as a therapeutic approach". I mean this literally. I can't imagine it. What else could they have said that would be more direct aside from HBOT will cure you 100% guaranteed?

mirekrusin 5y ago

Maybe 56 died?