4 ms·
I skimmed the article for the pertinent table. Three observations: 1. There is no control group. Since people get better with cognitive tests with practice, we
by parliowl 5y ago
I skimmed the article for the pertinent table. Three observations:
1. There is no control group. Since people get better with cognitive tests with practice, we don’t know whether these improvements were due to the treatment or just the normal practice effect. (Though if I were a betting person, my money is on the latter.)
2. Cognitive tests are not noiseless. The same person may vary by a few points between repeated tests. The modest improvements on most tests found here might not be above this noise floor.
3. The most impressive result (memory) is almost certainly regression to the mean. That is, the group did unusually poorly on this task in the first session by chance, then they performed more typically in the second, which looks falsely like a big improvement.
Basically, I think this paper provides no meaningful evidence of cognitive improvement in humans. I can’t speak about the mice stuff, that might be fine.
- jinto36 5y agoWhile a control group is essential to determine if the effect is really due to the treatment or to practice, if they did the mouse study first and expected a positive effect, then they might have felt pressure to only have a human treatment group in what looks like a pilot trial. Maybe their original proposal included a control group, but their institutional review board decided against it? I'm a little surprised they wouldn't have done a larger secondary trial before publishing, the impact would have been much greater if the effect was still observed in a larger group, even if they still did not have a control group- or perhaps had an alternative treatment schedule (shorter exposure times- is treatment duration correlated with degree of improvement? Then you have less of a moral dilemma of denying participants of a useful treatment while still getting some data that would indicate the treatment itself actually does something). (working on a PhD in genetics)
- simiones 5y agoI am not a practitioner, so I have no idea if this is actually plausible, but it would seem absurd to me for an experimental, out-of-left-field treatment for Alzheimers, in a <10 person study, could be denied a control group on ethical grounds. There are tens if not hundreds of millions of Alzheimer's patients not receiving this potential treatment already.
- lthornberry 5y agoThis. There is basically zero chance that an ethics review would see this as a case in which a control group is unethical. I have been through plenty of IRB processes. Resource constraints are a much more plausible explanation - running the study with a control group would cost twice as much (in time as well as money). This is either a pilot to apply for funding to run a better-controlled trial, or it's just not scientifically serious. Both are very plausible, in my experience.
- jinto36 5y agoThanks for the perspective, I have only done data analysis after the data is generated, and never have actually had to deal with an IRB directly for the design of a study.
- parliowl 5y agoI don’t want to speculate about why they designed their study this way or that way, because who knows. I do feel confident in saying that Scientifically, the human data in this study are largely uninformative.
- reacweb 5y agoYes control group is very important. This link might interest you https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/?fbclid=IwAR3mWDWkCcTwVyrOuUOAcpPF_YH7Xwk8duuyI1qIiduZ4I7E6m0OJyNyJck https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/?fbclid=...