12 ms·

I am a neurologist. this is actually the exact opposite of what you should care about, which is "does it work to treat ___" In this trial, and 99% of other tri

I am a neurologist. this is actually the exact opposite of what you should care about, which is "does it work to treat ___" In this trial, and 99% of other trials, there is no mechanism provided by the study, only a measurement of causality (which is the same thing as effectiveness). The authors may think they know based on other, prior, basic science research (as they are postulating here), but it does not affect the conclusions of the trial ( whether they do or don't (or even if they are wrong). Obviously it's intellectually satisfying to understand WHY, but it's not as important as "is it true?"

bashcoder 9y ago
In your view, does this study fall short of answering the “is it true?” question?
randomsearch 9y ago
The problem with “is this true?” Investigations is that there are tons of potential pitfalls and a combination of experimental limitations, poor academic incentives, and insufficient statistical and scientific training make this very hard to do well. I think it’s bad science, that will one day retrospectively be viewed as invalid scientific method. I think the way to combat this is to avoid studies that solely attempt to demonstrate an effect, but to complement that part of an investigation with additional studies that demonstrate a plausible explanation. This reduces the risk of false positives and increases our confidence in the result. The additional studies could take the form (for example) of deliberately intervening such that the effect should be manipulated by the intervention in a predictable and measurable way. Essentially, come at the hypothesis “from another angle”.
- deleted 9y ago
  [deleted]
- jeffdavis 9y ago
  I somewhat agree, but "is it true?" is all that science can answer. A mechanism is really just a narrower example without irrelevant distractions or confounding variables. Kind of like the difference between the bug reports: "run this complex workload on this specific machine and it will orobably crash after a few hours" versus "run these three commands when the cache is cold and it crashes".
  - fleitz 9y ago
    Actually all science can really answer is 'is it false?'.
  - jjgreen 9y ago
    I find it odd that the sibling reply by fleitz (all that science can say is "is it false"), which just a precis of Popperian philosophy of science, has been marked dead. Not a view that I share, but seems uncontroversial.
    - pbhjpbhj 9y ago
      Agreed, indeed my immediate response to that comment was "that's not how science works, it's falsificationist". What is your view on science, do you think it can create - and verify - a true result? Whither axiomatic underpinning in such a view?
      - jjgreen 9y ago
        Well thanks for asking. I think it's clear that science gets us truth, but that the mechanism for that complex social activity can't be reduced to a few axioms or principles; Feyerabend gives a detailed description [1] of how bad a scientist Galileo was (as judged by modern theories of what science is), yet he was right, and he advanced truth. [1] Paul Feyerabend, Against Method (1975)
        pbhjpbhj 9y ago
        And thanks for responding ... Heliocentrism isn't truth, it's opinion [that's my opinion, lol!!]. Einstein's relativity makes heliocentrism false in demanding that there is no aether and so no special points in space. Science provides our understanding of a best current description of reality - the whole point of scientific endeavour is to demonstrate we were wrong and that our understanding/description was flawed. Moreover it relies on maths, which is provably unable to verify it's own consistency and completeness. Furthermore it's an axiomatic system that produces a probabilistic framework that gives us little to hang our conception of reality on [what I'm saying is our perception is not particularly informed by physical reality, we have to subdue our instinctive perceptions to work with what amounts to our best, most consistent, physics view of reality]. Science gives us as much truth as we can see in a well lit test-tube.
- dumbneurologist 9y ago
  there are tons of potential pitfalls and a combination of experimental limitations you provided some, which are at least partially addressed by the study design. (I don't think it's your point, but let me agree that "the scientist can cheat" is always true). poor academic incentives, this is addressed because it is double-blind insufficient statistical and scientific training this is (partially) addressed by peer review, and the availability of the data. you can't make this general criticism of the paper at hand: find the place where the design or statistics used were in error. I think it’s bad science, that will one day retrospectively be viewed as invalid scientific method. that's a claim without any evidence to back it p. I think the way to combat this is to avoid studies that solely attempt to demonstrate an effect, but to complement that part of an investigation with additional studies that demonstrate a plausible explanation. This reduces the risk of false positives and increases our confidence in the result. If you have a double-blind, placebo-controlled, randomized clinical trial then what can the mechanism possibly add? Is an effective medication less effective because we don't understand how it works? If you want an amazing example, read about the mechanism of action of general anesthesia: we have many ideas about how it works, but nobody really knows. Nobody gets caught up by this when their appendix ruptures. I hope, for your sake, that you will lower your standards in the event that you need surgery. The additional studies could take the form (for example) of deliberately intervening such that the effect should be manipulated by the intervention in a predictable and measurable way. Essentially, come at the hypothesis “from another angle”. If you have a properly designed and executed randomized, double blind, placebo controlled trial, then you have measured the effect of the intervention, and only the intervention. You can repeat it in order to gain confidence, but you have to decide when the diminishing returns of "more confidence" are not worth your study dollar. And for curcurmin, which is cheap and has low side effects... it's probably not that far away. edit: fixed formatting
  - electrograv 9y ago
    >> I think it’s bad science, that will one day retrospectively be viewed as invalid scientific method. > that's a claim without any evidence to back it [u]p. You are correct. The irony is rich here. I always welcome a good argument (whether in dispute or affirmation of an issue), but just saying "I think <XYZ>" doesn't make it so. Reading through this thread, all I see is the common HN Default Skepticism / Negativity -- but more so; skepticism with less scientific rigor than the work being criticized.
pjmorris 9y ago
I am having trouble accepting your argument. Judea Pearl presents [0] the debate over whether smoking causes cancer as an illustration for why correlation is not sufficient. Two plausible, contradictory, models ('smoking causes cancer', 'no it doesn't, something else does') could not be settled by observing correlations. The debate was resolved through the addition of an explanatory mechanism, and a test of the mechanism's influence. Pearl, and others, have worked on the mathematical machinery necessary to examine this kind of causality. [0] http://singapore.cs.ucla.edu/LECTURE/lecture_sec3.htm http://singapore.cs.ucla.edu/LECTURE/lecture_sec3.htm
- nabla9 9y ago
  This study established causality with double blind test and thee effects size was really big.
- comstock 9y ago
  Unfortunately, we don’t understand the exact mechanism for many (most?) available medication. Correlation != causation. Which is why you need to do randomized testing and try and account for other causative factors. But if you give a bunch of people a medication and they stop dying, you can pretty much say this is valuable and start giving it to people.
- lern_too_spel 9y ago
  The issue that you're raising does not apply for randomized treatment.
- dumbneurologist 9y ago
  none of this applies to a double-blind, placebo-controlled, randomized clinical trial (which is what this article is). the reason it's the gold standard for medicine is because the only thing that is different in aggregate between the groups is the intervention. Therefore, it is the strongest possible evidence of causality we have, and does say something about causality (rather than correlation). It's important to replicate, but the points made by Pearl are not relevant here and the mechanism is moot when you have a causal link shown by RCT.
chiefalchemist 9y ago
OK. But without knowing why how can you be certain it's cause and not correlation? Especially in such a small trial?
- anentropic 9y ago
  isn't it because the control and treatment groups are randomised from the same pool?
  - chiefalchemist 9y ago
    It can still be correlation. It's possible you're just attributing something to an attribute you're tracking but that's just coincidence. Without a sense of why, you can't be certain about any conclusions.
    - anentropic 9y ago
      in this case they seemed to expect an effect on the amyloid plaques which cause Alzheimers and dementia (i.e. they bothered to do brain scans, not just cognitive tests)
deleted 9y ago
[deleted]
tremendulo 9y ago
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/
- brailsafe 9y ago
  Quite funny, had to reflect on my own bias a bit. I still think it's important to be critical of purely observational or anecdotal results when tested evidence driven medicine does exist. For example, suggesting a cancer patient abstain from radiation therapy, offering the alternative of cayenne pepper smoothies.
  - deleted 9y ago
    [deleted]
  - trhway 9y ago
    haven't seen cayenne pepper studies. While for example drinking baking soda - https://www.ncbi.nlm.nih.gov/pubmed/19276390 https://www.ncbi.nlm.nih.gov/pubmed/19276390 significantly slowing cancer spread with for example 4 months survival result - less than 40% of controls and more than 80% of baking soda drinker mice https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2834485/figure/F1/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2834485/figure/... The study has all the statistical bells and whistles. And its result perfectly matches our understanding of cancer spread process - local overproduction of acid from glucose aerobic glycolysis (dominating way of glucose consumption by cancer cells) results in dissolution of the ECM (which is mostly collagen) - and the probable mechanism of action of the baking soda on it.
staunch 9y ago
)
zaptheimpaler 9y ago
Obviously you don't understand HN.. everyone here is a "very smart software engineer" and knows far better than a simpleton like you with your decades of medical training and experience.
- cwyers 9y ago
  There are professional statisticians who believe the same as this HN cult. http://andrewgelman.com/2017/01/30/no-guru-no-method-no-teacher-just-nature-garden-forking-paths/ http://andrewgelman.com/2017/01/30/no-guru-no-method-no-teac...
- torstenvl 9y ago
  God forbid that anyone with a mathematical inclination begin to question study methodology... https://mobile.nytimes.com/2008/04/08/science/08tier.html https://mobile.nytimes.com/2008/04/08/science/08tier.html
- wavegeek 9y ago
  > decades of medical training and experience http://slatestarcodex.com/2013/12/17/statistical-literacy-among-doctors-now-lower-than-chance/ http://slatestarcodex.com/2013/12/17/statistical-literacy-am... Ask your local doctor next time you visit to tell you what a P value means. The medical literature is a tragicomedy of statistical incompetence and malpractice. Specifically on this study it has a lot of red flags: not pre-registered so p hacking can be assumed, industry sponsored which studies show greatly skews results, small numbers, unrealistically high effect sizes.
  - WalterSear 9y ago
    It was pre-registered. However, they did a battery of tests, and only report on a few. This is a method of cheating on the pre-registry process.
- jeffdavis 9y ago
  The neurologist wants what's best for his patients today, which is to know whether a treatment is likely to work or not. Others may value the mechanism more for very legitimate reasons. For instance, if you underatand the mechanism it leads to further research that could utilize a related mechanism or perhaps see potential side effects that would otherwise be hard to identify. You really need both. "Is it true?" is a great starting place and oofers some treatments, but discovering the mechanism gives potentially a whole new class of treatments.
cryptonector 9y ago
As far as neurology and psychiatry go, if we're not asking "how?", it's because we're very far from comprehending. Our brains are exceedingly complicated. If two neurons can ride a bicycle, imagine the complexity arising in a system with billions and billions. More generally, however, a plausible mechanism is a great aid in determining the potential value of further researching a correlation because as we all know, correlation != causation, but correlation + mechanism pretty much is causation, which means correlation + plausible mechanism is very interesting. It's not wrong to ask "how?" here, though it might not yield much given the nature of the field. Certainly, anti-inflammatory properties can plausibly have positive effects on the psyche, so that's a mechanism worth some looking into.
nabla9 9y ago
Can you comment on the practical improvements reported in this study? For example, How meaningful is the 20.3 point difference in Buschke SRT test? Is it noticeable in daily life, or just something detected in testing.

bashcoder 9y ago

In your view, does this study fall short of answering the “is it true?” question?

randomsearch 9y ago

The problem with “is this true?” Investigations is that there are tons of potential pitfalls and a combination of experimental limitations, poor academic incentives, and insufficient statistical and scientific training make this very hard to do well. I think it’s bad science, that will one day retrospectively be viewed as invalid scientific method. I think the way to combat this is to avoid studies that solely attempt to demonstrate an effect, but to complement that part of an investigation with additional studies that demonstrate a plausible explanation. This reduces the risk of false positives and increases our confidence in the result. The additional studies could take the form (for example) of deliberately intervening such that the effect should be manipulated by the intervention in a predictable and measurable way. Essentially, come at the hypothesis “from another angle”.

deleted 9y ago
[deleted]
jeffdavis 9y ago
I somewhat agree, but "is it true?" is all that science can answer. A mechanism is really just a narrower example without irrelevant distractions or confounding variables. Kind of like the difference between the bug reports: "run this complex workload on this specific machine and it will orobably crash after a few hours" versus "run these three commands when the cache is cold and it crashes".
- fleitz 9y ago
  Actually all science can really answer is 'is it false?'.
- jjgreen 9y ago
  I find it odd that the sibling reply by fleitz (all that science can say is "is it false"), which just a precis of Popperian philosophy of science, has been marked dead. Not a view that I share, but seems uncontroversial.
  - pbhjpbhj 9y ago
    Agreed, indeed my immediate response to that comment was "that's not how science works, it's falsificationist". What is your view on science, do you think it can create - and verify - a true result? Whither axiomatic underpinning in such a view?
    - jjgreen 9y ago
      Well thanks for asking. I think it's clear that science gets us truth, but that the mechanism for that complex social activity can't be reduced to a few axioms or principles; Feyerabend gives a detailed description [1] of how bad a scientist Galileo was (as judged by modern theories of what science is), yet he was right, and he advanced truth. [1] Paul Feyerabend, Against Method (1975)
      - pbhjpbhj 9y ago
        And thanks for responding ... Heliocentrism isn't truth, it's opinion [that's my opinion, lol!!]. Einstein's relativity makes heliocentrism false in demanding that there is no aether and so no special points in space. Science provides our understanding of a best current description of reality - the whole point of scientific endeavour is to demonstrate we were wrong and that our understanding/description was flawed. Moreover it relies on maths, which is provably unable to verify it's own consistency and completeness. Furthermore it's an axiomatic system that produces a probabilistic framework that gives us little to hang our conception of reality on [what I'm saying is our perception is not particularly informed by physical reality, we have to subdue our instinctive perceptions to work with what amounts to our best, most consistent, physics view of reality]. Science gives us as much truth as we can see in a well lit test-tube.
dumbneurologist 9y ago
there are tons of potential pitfalls and a combination of experimental limitations you provided some, which are at least partially addressed by the study design. (I don't think it's your point, but let me agree that "the scientist can cheat" is always true). poor academic incentives, this is addressed because it is double-blind insufficient statistical and scientific training this is (partially) addressed by peer review, and the availability of the data. you can't make this general criticism of the paper at hand: find the place where the design or statistics used were in error. I think it’s bad science, that will one day retrospectively be viewed as invalid scientific method. that's a claim without any evidence to back it p. I think the way to combat this is to avoid studies that solely attempt to demonstrate an effect, but to complement that part of an investigation with additional studies that demonstrate a plausible explanation. This reduces the risk of false positives and increases our confidence in the result. If you have a double-blind, placebo-controlled, randomized clinical trial then what can the mechanism possibly add? Is an effective medication less effective because we don't understand how it works? If you want an amazing example, read about the mechanism of action of general anesthesia: we have many ideas about how it works, but nobody really knows. Nobody gets caught up by this when their appendix ruptures. I hope, for your sake, that you will lower your standards in the event that you need surgery. The additional studies could take the form (for example) of deliberately intervening such that the effect should be manipulated by the intervention in a predictable and measurable way. Essentially, come at the hypothesis “from another angle”. If you have a properly designed and executed randomized, double blind, placebo controlled trial, then you have measured the effect of the intervention, and only the intervention. You can repeat it in order to gain confidence, but you have to decide when the diminishing returns of "more confidence" are not worth your study dollar. And for curcurmin, which is cheap and has low side effects... it's probably not that far away. edit: fixed formatting
- electrograv 9y ago
  >> I think it’s bad science, that will one day retrospectively be viewed as invalid scientific method. > that's a claim without any evidence to back it [u]p. You are correct. The irony is rich here. I always welcome a good argument (whether in dispute or affirmation of an issue), but just saying "I think <XYZ>" doesn't make it so. Reading through this thread, all I see is the common HN Default Skepticism / Negativity -- but more so; skepticism with less scientific rigor than the work being criticized.

pjmorris 9y ago

I am having trouble accepting your argument. Judea Pearl presents [0] the debate over whether smoking causes cancer as an illustration for why correlation is not sufficient. Two plausible, contradictory, models ('smoking causes cancer', 'no it doesn't, something else does') could not be settled by observing correlations. The debate was resolved through the addition of an explanatory mechanism, and a test of the mechanism's influence. Pearl, and others, have worked on the mathematical machinery necessary to examine this kind of causality. [0] http://singapore.cs.ucla.edu/LECTURE/lecture_sec3.htm http://singapore.cs.ucla.edu/LECTURE/lecture_sec3.htm

nabla9 9y ago
This study established causality with double blind test and thee effects size was really big.
comstock 9y ago
Unfortunately, we don’t understand the exact mechanism for many (most?) available medication. Correlation != causation. Which is why you need to do randomized testing and try and account for other causative factors. But if you give a bunch of people a medication and they stop dying, you can pretty much say this is valuable and start giving it to people.
lern_too_spel 9y ago
The issue that you're raising does not apply for randomized treatment.
dumbneurologist 9y ago
none of this applies to a double-blind, placebo-controlled, randomized clinical trial (which is what this article is). the reason it's the gold standard for medicine is because the only thing that is different in aggregate between the groups is the intervention. Therefore, it is the strongest possible evidence of causality we have, and does say something about causality (rather than correlation). It's important to replicate, but the points made by Pearl are not relevant here and the mechanism is moot when you have a causal link shown by RCT.

chiefalchemist 9y ago

OK. But without knowing why how can you be certain it's cause and not correlation? Especially in such a small trial?

anentropic 9y ago
isn't it because the control and treatment groups are randomised from the same pool?
- chiefalchemist 9y ago
  It can still be correlation. It's possible you're just attributing something to an attribute you're tracking but that's just coincidence. Without a sense of why, you can't be certain about any conclusions.
  - anentropic 9y ago
    in this case they seemed to expect an effect on the amyloid plaques which cause Alzheimers and dementia (i.e. they bothered to do brain scans, not just cognitive tests)

deleted 9y ago

[deleted]

tremendulo 9y ago

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/

brailsafe 9y ago
Quite funny, had to reflect on my own bias a bit. I still think it's important to be critical of purely observational or anecdotal results when tested evidence driven medicine does exist. For example, suggesting a cancer patient abstain from radiation therapy, offering the alternative of cayenne pepper smoothies.
- deleted 9y ago
  [deleted]
- trhway 9y ago
  haven't seen cayenne pepper studies. While for example drinking baking soda - https://www.ncbi.nlm.nih.gov/pubmed/19276390 https://www.ncbi.nlm.nih.gov/pubmed/19276390 significantly slowing cancer spread with for example 4 months survival result - less than 40% of controls and more than 80% of baking soda drinker mice https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2834485/figure/F1/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2834485/figure/... The study has all the statistical bells and whistles. And its result perfectly matches our understanding of cancer spread process - local overproduction of acid from glucose aerobic glycolysis (dominating way of glucose consumption by cancer cells) results in dissolution of the ECM (which is mostly collagen) - and the probable mechanism of action of the baking soda on it.

staunch 9y ago

)

zaptheimpaler 9y ago

Obviously you don't understand HN.. everyone here is a "very smart software engineer" and knows far better than a simpleton like you with your decades of medical training and experience.

cwyers 9y ago
There are professional statisticians who believe the same as this HN cult. http://andrewgelman.com/2017/01/30/no-guru-no-method-no-teacher-just-nature-garden-forking-paths/ http://andrewgelman.com/2017/01/30/no-guru-no-method-no-teac...
torstenvl 9y ago
God forbid that anyone with a mathematical inclination begin to question study methodology... https://mobile.nytimes.com/2008/04/08/science/08tier.html https://mobile.nytimes.com/2008/04/08/science/08tier.html
wavegeek 9y ago
> decades of medical training and experience http://slatestarcodex.com/2013/12/17/statistical-literacy-among-doctors-now-lower-than-chance/ http://slatestarcodex.com/2013/12/17/statistical-literacy-am... Ask your local doctor next time you visit to tell you what a P value means. The medical literature is a tragicomedy of statistical incompetence and malpractice. Specifically on this study it has a lot of red flags: not pre-registered so p hacking can be assumed, industry sponsored which studies show greatly skews results, small numbers, unrealistically high effect sizes.
- WalterSear 9y ago
  It was pre-registered. However, they did a battery of tests, and only report on a few. This is a method of cheating on the pre-registry process.
jeffdavis 9y ago
The neurologist wants what's best for his patients today, which is to know whether a treatment is likely to work or not. Others may value the mechanism more for very legitimate reasons. For instance, if you underatand the mechanism it leads to further research that could utilize a related mechanism or perhaps see potential side effects that would otherwise be hard to identify. You really need both. "Is it true?" is a great starting place and oofers some treatments, but discovering the mechanism gives potentially a whole new class of treatments.

cryptonector 9y ago

As far as neurology and psychiatry go, if we're not asking "how?", it's because we're very far from comprehending. Our brains are exceedingly complicated. If two neurons can ride a bicycle, imagine the complexity arising in a system with billions and billions. More generally, however, a plausible mechanism is a great aid in determining the potential value of further researching a correlation because as we all know, correlation != causation, but correlation + mechanism pretty much is causation, which means correlation + plausible mechanism is very interesting. It's not wrong to ask "how?" here, though it might not yield much given the nature of the field. Certainly, anti-inflammatory properties can plausibly have positive effects on the psyche, so that's a mechanism worth some looking into.

nabla9 9y ago

Can you comment on the practical improvements reported in this study? For example, How meaningful is the 20.3 point difference in Buschke SRT test? Is it noticeable in daily life, or just something detected in testing.