6 ms·

> Small study Almost entirely irrelevant, provided it's properly powered--and based on the effect size and p-values, it is. Furthermore, for an 18 month trial,

by sov 9y ago

> Small study Almost entirely irrelevant, provided it's properly powered--and based on the effect size and p-values, it is. Furthermore, for an 18 month trial, 40 people is larger than most. > probably not preregistered I mean, true, and we should definitely promote pre-reg, but, 1. so few are preregistered that I'm not sure it's prudent to disregard a study based on this and 2. it's an academic study, neither a clinical trial nor a public-health decree--not a typical study type that necessitates pre-registration. > Likely Publication Bias or p-hacking. What? You're arrived at this conclusion based on... the fact that it's "small" and you couldn't find preregistration about it? > I'm starting to get interested when it's been independently replicated. Yeah, if only there were another trial that showed a positive cognitive effect of bioavailable curcumin... https://www.ncbi.nlm.nih.gov/pubmed/25277322 https://www.ncbi.nlm.nih.gov/pubmed/25277322 I mean, the title of this post is by far the most wrong thing about this article, as... 1. it's not turmeric, it's curcumin, which is about 2% of turmeric by mass (so, you'd need to eat ~4.5g turmeric daily to match study doses). 2. curcumin doesn't even have any reasonable bioavilability by itself, so they're using a bioavailable analogue.

andrewjl 9y ago
What prevents researchers from running studies like these on the order of thousands of individuals, as opposed to 40 or 100?
- shanedrgn 9y ago
  Funding
- sov 9y ago
  A lot of things that basically boil down to budget, optimization and space available. If you have a drug that you believe you can show a strong effect with the effect group at ~15 people, maybe you run the trial with 40 (15 e.g., 15 control, plus wiggle room for drop-outs over the 18-month course and extra certainty). Check the last few paragraphs--"The authors thank Shayna Greenberg, Dev Darshan Khalsa, and Anya Rosensteel for help in recruitment, data management, and study coordination; John Williams, University of California, Los Angeles, Nuclear Medicine Clinic, for performing the PET scans; and Vladimir Kepe, Cleveland Clinic, for developing the FDDNP-PET analysis procedures for proteinopathies in patients with neurodegenerative diseases." Add all that to the effort involved in the actual chemical extraction of the bioavailable curcumin analogue and it just doesn't make sense to add any more people than you need.
  - andrewjl 9y ago
    For the recruitment, data management, and study coordination piece, sounds like a great area to optimize with software. I'm sure big pharma will also happily shell out for something to help them manage their own clinical trials. Academia PIs can then get the same stuff at a lower rate. Edit: Heck, you can even lease top grade medical space in all major metro areas and offer it as part of the package in a WeWork style arrangement. I'm sure they are tons of nuances to rolling something like this out, but it sounds like it can cut costs substantially.
- donquichotte 9y ago
  Money.
- e12e 9y ago
  If they're marketing their own drug (as implied by other comments - I don't know), you might also ask what stops them from running the studio on a 100 groups of 40 patients, and picking the "best" for publication... (other than ethics, obviously)...
  - pbhjpbhj 9y ago
    Cheaper to run it on 100 and select the best 40 to report in your study.
- yufeng66 9y ago
  As others have pointed out. Its about money. If the compound is expected to be very profitable, then it make sense to invest a lot money to do a large scale study. On the other hand, if it is a natural compound that you can't really patent, the profitability will be much lower. It doesn't make sense for a commercial entity to do a large scale study. If the compound is important enough, a government entity will sometimes invest the money to study it.
brfox 9y ago
Last paragraph: The University of California, Los Angeles, owns a U.S. patent (6,274,119) entitled “Methods for Labeling ß-Amyloid Plaques and Neurofibrillary Tangles”, which has been licensed to TauMark, LLC. Drs. Small, Satyamurthy, Huang, and Barrio are among the inventors and have financial interest in TauMark, LLC. Dr. Small also reports having served as an advisor to and/or having received lecture fees from Allergan, Argentum, Axovant, Cogniciti, Forum Pharmaceuticals, Herbalife, Janssen, Lundbeck, Lilly, Novartis, Otsuka, and Pfizer. Dr. Heber reports receiving consulting fees from Herbalife, and the McCormick Science Institute. The manufacturer of Theracurmin, Theravalues Corporation, provided the Theracurmin and placebo for the trial, funds for laboratory testing of blood curcumin levels, and funds for Dr. Small's travel to the 2017 Alzheimer's Association International Conference for presentation of the findings.
- sov 9y ago
  If that were the gp's argument, I'd have no problem with it (heck--I'd agree with most of it).
nabla9 9y ago
Thank you for your informed comment. I don't' work in the field so I find it hard to interpret these studies. What's your opinion about the size of improvements in practice? For example, How meaningful is the 20.3 point difference in Buschke SRT test? Cohen's d (mean difference divided by SD) above 0.60 with the memory or attention seems good (73 % of the people who take the dose get above the mean test score). But difference that is detectable in a test is not telling me how big the effect would be in the daily performance unless I'm familiar with the test.
eni 9y ago
it's not turmeric, it's curcumin, which is about 2% of turmeric by mass (so, you'd need to eat ~4.5g turmeric daily to match study doses) Typical Indian cooking uses turmeric in pretty much everything. I wouldn't be surprised if daily consumption of turmeric is very close to the figures you mentioned.
- wastedhours 9y ago
  Interestingly, one of the recent fads I've been seeing has been for "turmeric lattes" - maybe can reach that level even if you're not keen on Indian cuisine?
  - eni 9y ago
    "turmeric lattes" :) My kitchen medicine for running nose, soar throat is hot milk with turmeric. Works for me most of the times.
  - vanderZwan 9y ago
    Apparently, eating more than a teaspoon of tumeric a day increases the risk of kidney stones though, so don't overdo it. I've been mixing it through my oatmeal, together with chilli-pepper and black pepper, a bit of cardamom, and a teaspoon of real cinnamon (the non-cumerin kind). While it feels like it works, I know that the placebo effect is probably much stronger than any real effect these foods have. Still, a placebo effect is still a real effect on my mood, so that's still a kind of health benefit I guess. If nothing else the spices help me wake up! Tangent regarding health food fads: these days I mainly use nutritionfacts.org[0] to determine which of those are actually supported by the latest nutrition research. It is the only health food/diet website that I know of that directly cites nutrition research and continuously scours the latest papers for new findings - sources are always linked, and quotes are directly lifted from the papers with no modification (so less likely to suffer from "stronger or opposite of what the paper actually says"-shenanigans often seen elsewhere). Some of the other things I tried out based on that website have too big of an effect to just be placebo: adding blue-berries to a meal really reduces sugar rushes/crashes in the hours after it[1]. Taking a table spoon of freshly broken flax seeds[2] does a lot to counter the rise in blood pressure due to my ADD medication (I measured it, plus I feel a lot less discomfort in my chest area and less jittery). Last time I mentioned that website here, it was in a discussion of which diets are healthy. Ironically, it was the only comment in the discussion that attracted multiples downvotes without explanation, while everyone else was sharing their unsourced opinions. [0] https://nutritionfacts.org/ https://nutritionfacts.org/ [1] https://nutritionfacts.org/video/green-smoothies-what-does-the-science-say/ https://nutritionfacts.org/video/green-smoothies-what-does-t... [2] https://nutritionfacts.org/video/flax-seeds-for-hypertension/ https://nutritionfacts.org/video/flax-seeds-for-hypertension...
martingoodson 9y ago
>> Small study >Almost entirely irrelevant, provided it's properly powered--and based on the effect size and p-values, it is. Sorry to state this so strongly but I feel it's important to point out that this is misinformation of the most egregious kind. There is no examination of statistical power in this article whatsoever so there is no basis for your claim that this study is properly powered. Large effect sizes are not evidence of adequate statistical power. Quite the opposite is true. E.g. See this article: https://www.nature.com/articles/nrn3475/figures/5 https://www.nature.com/articles/nrn3475/figures/5 'Effect inflation is worst for small, low-powered studies, which can only detect effects that happen to be large.'
- sideshowb 9y ago
  Am I right in thinking that statistical power is the true positive rate, so a test that always rejected the null hypothesis (whether correctly or incorrectly) would have the highest possible statistical power?
  - closed 9y ago
    No, statistical power as a concept is independent of the observed result of a test. It is widely misunderstood though, so often people try to use an observed result to justify low power. http://andrewgelman.com/2017/08/17/just-google-despite-limited-statistical-power/ http://andrewgelman.com/2017/08/17/just-google-despite-limit...
    - sideshowb 9y ago
      Is the wikipedia page on statistical power plain wrong then? "The power of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis (H0) when a specific alternative hypothesis (H1) is true. The statistical power ranges from 0 to 1, and as statistical power increases, the probability of making a type 2 error decreases. For a type 2 error probability of β, the corresponding statistical power is 1-β. " So according to that, if I always reject H0 whether rightly or wrongly, that means β=0 and power=1.
- 9y ago

by sov 9y ago

andrewjl 9y ago

What prevents researchers from running studies like these on the order of thousands of individuals, as opposed to 40 or 100?

shanedrgn 9y ago
Funding
sov 9y ago
A lot of things that basically boil down to budget, optimization and space available. If you have a drug that you believe you can show a strong effect with the effect group at ~15 people, maybe you run the trial with 40 (15 e.g., 15 control, plus wiggle room for drop-outs over the 18-month course and extra certainty). Check the last few paragraphs--"The authors thank Shayna Greenberg, Dev Darshan Khalsa, and Anya Rosensteel for help in recruitment, data management, and study coordination; John Williams, University of California, Los Angeles, Nuclear Medicine Clinic, for performing the PET scans; and Vladimir Kepe, Cleveland Clinic, for developing the FDDNP-PET analysis procedures for proteinopathies in patients with neurodegenerative diseases." Add all that to the effort involved in the actual chemical extraction of the bioavailable curcumin analogue and it just doesn't make sense to add any more people than you need.
- andrewjl 9y ago
  For the recruitment, data management, and study coordination piece, sounds like a great area to optimize with software. I'm sure big pharma will also happily shell out for something to help them manage their own clinical trials. Academia PIs can then get the same stuff at a lower rate. Edit: Heck, you can even lease top grade medical space in all major metro areas and offer it as part of the package in a WeWork style arrangement. I'm sure they are tons of nuances to rolling something like this out, but it sounds like it can cut costs substantially.
donquichotte 9y ago
Money.
e12e 9y ago
If they're marketing their own drug (as implied by other comments - I don't know), you might also ask what stops them from running the studio on a 100 groups of 40 patients, and picking the "best" for publication... (other than ethics, obviously)...
- pbhjpbhj 9y ago
  Cheaper to run it on 100 and select the best 40 to report in your study.
yufeng66 9y ago
As others have pointed out. Its about money. If the compound is expected to be very profitable, then it make sense to invest a lot money to do a large scale study. On the other hand, if it is a natural compound that you can't really patent, the profitability will be much lower. It doesn't make sense for a commercial entity to do a large scale study. If the compound is important enough, a government entity will sometimes invest the money to study it.

brfox 9y ago

Last paragraph: The University of California, Los Angeles, owns a U.S. patent (6,274,119) entitled “Methods for Labeling ß-Amyloid Plaques and Neurofibrillary Tangles”, which has been licensed to TauMark, LLC. Drs. Small, Satyamurthy, Huang, and Barrio are among the inventors and have financial interest in TauMark, LLC. Dr. Small also reports having served as an advisor to and/or having received lecture fees from Allergan, Argentum, Axovant, Cogniciti, Forum Pharmaceuticals, Herbalife, Janssen, Lundbeck, Lilly, Novartis, Otsuka, and Pfizer. Dr. Heber reports receiving consulting fees from Herbalife, and the McCormick Science Institute. The manufacturer of Theracurmin, Theravalues Corporation, provided the Theracurmin and placebo for the trial, funds for laboratory testing of blood curcumin levels, and funds for Dr. Small's travel to the 2017 Alzheimer's Association International Conference for presentation of the findings.

sov 9y ago
If that were the gp's argument, I'd have no problem with it (heck--I'd agree with most of it).

nabla9 9y ago

Thank you for your informed comment. I don't' work in the field so I find it hard to interpret these studies. What's your opinion about the size of improvements in practice? For example, How meaningful is the 20.3 point difference in Buschke SRT test? Cohen's d (mean difference divided by SD) above 0.60 with the memory or attention seems good (73 % of the people who take the dose get above the mean test score). But difference that is detectable in a test is not telling me how big the effect would be in the daily performance unless I'm familiar with the test.

eni 9y ago

it's not turmeric, it's curcumin, which is about 2% of turmeric by mass (so, you'd need to eat ~4.5g turmeric daily to match study doses) Typical Indian cooking uses turmeric in pretty much everything. I wouldn't be surprised if daily consumption of turmeric is very close to the figures you mentioned.

wastedhours 9y ago
Interestingly, one of the recent fads I've been seeing has been for "turmeric lattes" - maybe can reach that level even if you're not keen on Indian cuisine?
- eni 9y ago
  "turmeric lattes" :) My kitchen medicine for running nose, soar throat is hot milk with turmeric. Works for me most of the times.
- vanderZwan 9y ago
  Apparently, eating more than a teaspoon of tumeric a day increases the risk of kidney stones though, so don't overdo it. I've been mixing it through my oatmeal, together with chilli-pepper and black pepper, a bit of cardamom, and a teaspoon of real cinnamon (the non-cumerin kind). While it feels like it works, I know that the placebo effect is probably much stronger than any real effect these foods have. Still, a placebo effect is still a real effect on my mood, so that's still a kind of health benefit I guess. If nothing else the spices help me wake up! Tangent regarding health food fads: these days I mainly use nutritionfacts.org[0] to determine which of those are actually supported by the latest nutrition research. It is the only health food/diet website that I know of that directly cites nutrition research and continuously scours the latest papers for new findings - sources are always linked, and quotes are directly lifted from the papers with no modification (so less likely to suffer from "stronger or opposite of what the paper actually says"-shenanigans often seen elsewhere). Some of the other things I tried out based on that website have too big of an effect to just be placebo: adding blue-berries to a meal really reduces sugar rushes/crashes in the hours after it[1]. Taking a table spoon of freshly broken flax seeds[2] does a lot to counter the rise in blood pressure due to my ADD medication (I measured it, plus I feel a lot less discomfort in my chest area and less jittery). Last time I mentioned that website here, it was in a discussion of which diets are healthy. Ironically, it was the only comment in the discussion that attracted multiples downvotes without explanation, while everyone else was sharing their unsourced opinions. [0] https://nutritionfacts.org/ https://nutritionfacts.org/ [1] https://nutritionfacts.org/video/green-smoothies-what-does-the-science-say/ https://nutritionfacts.org/video/green-smoothies-what-does-t... [2] https://nutritionfacts.org/video/flax-seeds-for-hypertension/ https://nutritionfacts.org/video/flax-seeds-for-hypertension...

martingoodson 9y ago

>> Small study >Almost entirely irrelevant, provided it's properly powered--and based on the effect size and p-values, it is. Sorry to state this so strongly but I feel it's important to point out that this is misinformation of the most egregious kind. There is no examination of statistical power in this article whatsoever so there is no basis for your claim that this study is properly powered. Large effect sizes are not evidence of adequate statistical power. Quite the opposite is true. E.g. See this article: https://www.nature.com/articles/nrn3475/figures/5 https://www.nature.com/articles/nrn3475/figures/5 'Effect inflation is worst for small, low-powered studies, which can only detect effects that happen to be large.'

sideshowb 9y ago
Am I right in thinking that statistical power is the true positive rate, so a test that always rejected the null hypothesis (whether correctly or incorrectly) would have the highest possible statistical power?
- closed 9y ago
  No, statistical power as a concept is independent of the observed result of a test. It is widely misunderstood though, so often people try to use an observed result to justify low power. http://andrewgelman.com/2017/08/17/just-google-despite-limited-statistical-power/ http://andrewgelman.com/2017/08/17/just-google-despite-limit...
  - sideshowb 9y ago
    Is the wikipedia page on statistical power plain wrong then? "The power of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis (H0) when a specific alternative hypothesis (H1) is true. The statistical power ranges from 0 to 1, and as statistical power increases, the probability of making a type 2 error decreases. For a type 2 error probability of β, the corresponding statistical power is 1-β. " So according to that, if I always reject H0 whether rightly or wrongly, that means β=0 and power=1.
9y ago