5 ms·

when i read this i saw no mention of a control group?

by gameswithgo 8y ago

peteretep 8y ago
No, and the last sentence acknowledges that this should now be repeated more rigorously
- amelius 8y ago
  A bit strange that this was accepted in "Nature" then (?)
  - dgacmu 8y ago
    Not really. It's common to accept papers based upon some function of Interesting & confidence, as long as the papers are very clear about their significance (or in this case, lack of a control). this kind of preliminary study is important in being able to make a case for funding a much larger formal study.
  - galangalalgol 8y ago
    It is a hot trend to correlate things to gut biome. Even so, the fact so many researchers are able to do so is quite interesting.
    - pvaldes 8y ago
      It has been hypotesized that autism increase in the last decades might be correlated with the increase in the use of pesticides. Early damage in nervous system that leads to a permanent particular architecture in the brain and all the stuff. Some gut bioma could theoretically be slightly better than other to deal with it. Some bacterias could be more efficient metabolizing some damaging products in our diet into other not so bad or even inocuous. Is teorethically possible. Both problems could be related and in this sense is a pretty interesting field. They could just give to an artificial gut bioma in a petri dish some diluted compounds, (or just give some autist children organic cultured fruits and keep it in a pesticide free environment) and see what happens
      - lordlimecat 8y ago
        Is there any evidence that the amount and types of pesticides used today are more damaging than those used 30 years ago? Or 60 years ago? Gut feeling: regulatory environment is a lot more strict these days, so we're probably a lot better off with today's pesticides than whatever straight-up toxic chemicals were used in the past. I bet if we started looking we'd see things like arsenic and similar used in the past.
        pvaldes 8y ago
        Don't need to be more damaging. Can be harmful in a different sense.
  - ramraj07 8y ago
    This is not Nature, this is "scientific report", which is not a bad journal but no Nature either.
    - jessriedel 8y ago
      Slight clarification: The journal is called Scientific Reports, which as you correctly emphasize is very different than the journal Nature even though they are owned by the same company. The articles are still called "articles" though.
    - azeotropic 8y ago
      It's basically Springer-Nature's answer to PLoS One. As long as it's not obviously crazy, anything goes. This doesn't look very convincing though. Without a no-treatment or sham treatment control, the results could be placebo effect, reversion to the mean on their metrics, or due to the antibiotic treatment rather than the transplanted gut bacteria, or a just a statistical fluke with this small population. One thing that stands out to me is the lack of difference in developmental age between 8 weeks and 2 years after treatment. I find it hard to understand how that would occur under the model where this treatment does something.
  - hyperbovine 8y ago
    Nature Scientific Reports, not the same.
    - jessriedel 8y ago
      In fact, "Nature" isn't even in the office journal title, unlike basically every other journal owned by the Springer Nature publishing company, e.g., Nature Communications, Nature Chemistry, Nature Methods. This highlights that it's considered to be a significantly lower tier in prestige, and the acceptance rates are much higher. Which is why I've published there :D
  - skosuri 8y ago
    Scientific reports is a nature family journal somewhat akin to Plos One (very little importance review). It should set some warning lights on.
  - avip 8y ago
    It wasn’t.
  - SubiculumCode 8y ago
    This is Nature Scientific Reports, a low impact journal.
strainer 8y ago
But the results are very strong. In the treatment group, almost 1 in 5 patients went from severe autistic symptoms to normal (clear). And almost 2 in 5 patients went from severe to mild. This begs the question, is any other treatment known to produce such strong results?
- gameswithgo 8y ago
  Given the method of measuring the results I'm not sure we can know anything about how strong the results are without a control group. It seems there is a deal of subjectivity in interpreting the results, and when this is the case crazy stuff can and does happen, regularly.
  - strainer 8y ago
    Can you clarify what you are saying about the method of measuring results?
    - teraflop 8y ago
      The study evaluated the severity of autism symptoms (using subjective questionnaires) before and after the treatment, but didn't compare those results to a control group of people who weren't treated the same way. It's like finding a couple dozen people with the flu, feeding them chicken soup, and then reporting that a month later they feel healthier, so the soup must have helped. Maybe it did, maybe it didn't, but you didn't measure that so you don't know.
      - strainer 8y ago
        I asked as an aside to the matter of control group - what is considered "given" about "the method of measurement" ? According to this professional medical trial, the recorded ASD scores are "based on the Childhood Autism Rating Scale (CARS) rated by a professional evaluator" You should have specific knowledge of that evaluation in order to criticize it as relying on "subjective questionnaires" and shouldn't introduce doubt by simply stating something as "given".
RobertRoberts 8y ago
Lot's of scientific studies have no control groups.
- lordlimecat 8y ago
  Not good ones. Any study attempting to show or imply causation needs to control for confounding variables; this is statistics 101. Usually you do this with an aptly named control group. If you don't have that control, you're left with "either my results are meaningful or else (very likely) there is a confounding factor." Great, that's not very useful.
  - RobertRoberts 8y ago
    Not a single study on the efficacy of vaccines has had a control group. When I asked why, I was told it was "unethical". So either, vaccine studies are "not good ones" as you say, or you perhaps have overlooked something?
    - gus_massa 8y ago
      LMGTFY It was difficult, but I got these two: (1954) Salk study of anti polio vaccine: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1114166/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1114166/ (2009) Pregnancy and infant results after anti human papillomavirus: https://www.ncbi.nlm.nih.gov/pubmed/19935017 https://www.ncbi.nlm.nih.gov/pubmed/19935017 There are probably more, but they are difficult to find.
      - RobertRoberts 8y ago
        Neither of these studies did they control for who was exposed to the actual disease. As far as I have found, the only experiments that did do this were completely unethical, and have not been repeated in modern western science: (that I know of) https://encyclopedia.ushmm.org/content/en/article/nazi-medical-experiments https://encyclopedia.ushmm.org/content/en/article/nazi-medic... http://www.auschwitz.dk/Bullenhuser.htm http://www.auschwitz.dk/Bullenhuser.htm

when i read this i saw no mention of a control group?

peteretep 8y ago

No, and the last sentence acknowledges that this should now be repeated more rigorously

amelius 8y ago
A bit strange that this was accepted in "Nature" then (?)
- dgacmu 8y ago
  Not really. It's common to accept papers based upon some function of Interesting & confidence, as long as the papers are very clear about their significance (or in this case, lack of a control). this kind of preliminary study is important in being able to make a case for funding a much larger formal study.
- galangalalgol 8y ago
  It is a hot trend to correlate things to gut biome. Even so, the fact so many researchers are able to do so is quite interesting.
  - pvaldes 8y ago
    It has been hypotesized that autism increase in the last decades might be correlated with the increase in the use of pesticides. Early damage in nervous system that leads to a permanent particular architecture in the brain and all the stuff. Some gut bioma could theoretically be slightly better than other to deal with it. Some bacterias could be more efficient metabolizing some damaging products in our diet into other not so bad or even inocuous. Is teorethically possible. Both problems could be related and in this sense is a pretty interesting field. They could just give to an artificial gut bioma in a petri dish some diluted compounds, (or just give some autist children organic cultured fruits and keep it in a pesticide free environment) and see what happens
    - lordlimecat 8y ago
      Is there any evidence that the amount and types of pesticides used today are more damaging than those used 30 years ago? Or 60 years ago? Gut feeling: regulatory environment is a lot more strict these days, so we're probably a lot better off with today's pesticides than whatever straight-up toxic chemicals were used in the past. I bet if we started looking we'd see things like arsenic and similar used in the past.
      - pvaldes 8y ago
        Don't need to be more damaging. Can be harmful in a different sense.
- ramraj07 8y ago
  This is not Nature, this is "scientific report", which is not a bad journal but no Nature either.
  - jessriedel 8y ago
    Slight clarification: The journal is called Scientific Reports, which as you correctly emphasize is very different than the journal Nature even though they are owned by the same company. The articles are still called "articles" though.
  - azeotropic 8y ago
    It's basically Springer-Nature's answer to PLoS One. As long as it's not obviously crazy, anything goes. This doesn't look very convincing though. Without a no-treatment or sham treatment control, the results could be placebo effect, reversion to the mean on their metrics, or due to the antibiotic treatment rather than the transplanted gut bacteria, or a just a statistical fluke with this small population. One thing that stands out to me is the lack of difference in developmental age between 8 weeks and 2 years after treatment. I find it hard to understand how that would occur under the model where this treatment does something.
- hyperbovine 8y ago
  Nature Scientific Reports, not the same.
  - jessriedel 8y ago
    In fact, "Nature" isn't even in the office journal title, unlike basically every other journal owned by the Springer Nature publishing company, e.g., Nature Communications, Nature Chemistry, Nature Methods. This highlights that it's considered to be a significantly lower tier in prestige, and the acceptance rates are much higher. Which is why I've published there :D
- skosuri 8y ago
  Scientific reports is a nature family journal somewhat akin to Plos One (very little importance review). It should set some warning lights on.
- avip 8y ago
  It wasn’t.
- SubiculumCode 8y ago
  This is Nature Scientific Reports, a low impact journal.

strainer 8y ago

But the results are very strong. In the treatment group, almost 1 in 5 patients went from severe autistic symptoms to normal (clear). And almost 2 in 5 patients went from severe to mild. This begs the question, is any other treatment known to produce such strong results?

gameswithgo 8y ago
Given the method of measuring the results I'm not sure we can know anything about how strong the results are without a control group. It seems there is a deal of subjectivity in interpreting the results, and when this is the case crazy stuff can and does happen, regularly.
- strainer 8y ago
  Can you clarify what you are saying about the method of measuring results?
  - teraflop 8y ago
    The study evaluated the severity of autism symptoms (using subjective questionnaires) before and after the treatment, but didn't compare those results to a control group of people who weren't treated the same way. It's like finding a couple dozen people with the flu, feeding them chicken soup, and then reporting that a month later they feel healthier, so the soup must have helped. Maybe it did, maybe it didn't, but you didn't measure that so you don't know.
    - strainer 8y ago
      I asked as an aside to the matter of control group - what is considered "given" about "the method of measurement" ? According to this professional medical trial, the recorded ASD scores are "based on the Childhood Autism Rating Scale (CARS) rated by a professional evaluator" You should have specific knowledge of that evaluation in order to criticize it as relying on "subjective questionnaires" and shouldn't introduce doubt by simply stating something as "given".

RobertRoberts 8y ago

Lot's of scientific studies have no control groups.

lordlimecat 8y ago
Not good ones. Any study attempting to show or imply causation needs to control for confounding variables; this is statistics 101. Usually you do this with an aptly named control group. If you don't have that control, you're left with "either my results are meaningful or else (very likely) there is a confounding factor." Great, that's not very useful.
- RobertRoberts 8y ago
  Not a single study on the efficacy of vaccines has had a control group. When I asked why, I was told it was "unethical". So either, vaccine studies are "not good ones" as you say, or you perhaps have overlooked something?
  - gus_massa 8y ago
    LMGTFY It was difficult, but I got these two: (1954) Salk study of anti polio vaccine: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1114166/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1114166/ (2009) Pregnancy and infant results after anti human papillomavirus: https://www.ncbi.nlm.nih.gov/pubmed/19935017 https://www.ncbi.nlm.nih.gov/pubmed/19935017 There are probably more, but they are difficult to find.
    - RobertRoberts 8y ago
      Neither of these studies did they control for who was exposed to the actual disease. As far as I have found, the only experiments that did do this were completely unethical, and have not been repeated in modern western science: (that I know of) https://encyclopedia.ushmm.org/content/en/article/nazi-medical-experiments https://encyclopedia.ushmm.org/content/en/article/nazi-medic... http://www.auschwitz.dk/Bullenhuser.htm http://www.auschwitz.dk/Bullenhuser.htm