5 ms·

> odds were tripled for autism (OR = 3.1; 95% CI: 1.02, 9.7) This is scientific malpractice! The most ridiculous confidence interval I've ever seen! 1.02 to 9.

> odds were tripled for autism (OR = 3.1; 95% CI: 1.02, 9.7) This is scientific malpractice! The most ridiculous confidence interval I've ever seen! 1.02 to 9.7, reported as "tripled", seriously? And of course the data is non-blinded, self-reported survey responses recalling events that occurred many years ago, and the analysis is not preregistered and splits the cohort in an arbitrary way to eke out so-called "statistical significance" (by the slimmest imaginable margin, 1.02 > 1.00, just barely). How can this dreck be published? Everyone involved should be sanctioned. And everyone who took this headline at face value should seriously reconsider their approach to consuming science news.

dkasper 3y ago
Yeah, I’m gonna need to see some replication with stronger methodology than self reporting and better confidence interval before I put much weight on this study.
- xwdv 3y ago
  [flagged]
  - beebmam 3y ago
    Why discourage people from any behavior without evidence or good reason?
    - xwdv 3y ago
      Soda is generally very bad for one’s overall health. Although a person might not care about themselves enough to stop, perhaps they will care if their child is at risk.
  - jeremyjh 3y ago
    Yes, misleading people for their own good is such a great strategy, it always works out just fine.
  - deleted 3y ago
    [deleted]
  - Avshalom 3y ago
    what is a harmful soda?
  - achandlerwhite 3y ago
    "harmful"... citation needed
  - nikau 3y ago
    > Why not put weight on Well they are drinking diet soda...
  - ftxbro 3y ago
    [flagged]
ToValueFunfetti 3y ago
They're at least acknowledging obesity as a confound, but I don't see any reference to sugar sodas or caffeine in a Ctrl-F + skim.
- adrr 3y ago
  I didn’t see any reference to controlling for age which is has been shown to be one the biggest factors in autism.
deleted 3y ago
[deleted]
tpmx 3y ago
The r/science discussion of this paper 3 days ago is interesting: https://old.reddit.com/r/science/comments/16t4eyg/drinking_diet_sodas_and_aspartamesweetened/ https://old.reddit.com/r/science/comments/16t4eyg/drinking_d...
AdmiralAsshat 3y ago
What was it Kurt Vonnegut said? "Pity the reader." The average person has little to no understanding of statistics. The study is coming from a (seemingly) reputable university source. What reason would the uninformed have to be skeptical?
- tpmx 3y ago
  It's published in Nutrients. https://en.wikipedia.org/wiki/MDPI#Resignations_of_editors https://en.wikipedia.org/wiki/MDPI#Resignations_of_editors In August 2018, 10 senior editors (including the editor-in-chief) of the journal Nutrients resigned, alleging that MDPI forced the replacement of the editor-in-chief because of his high editorial standards and for resisting pressure to "accept manuscripts of mediocre quality and importance.
- dimal 3y ago
  Yet, on other topics, people have “trust science” drilled into them, and are made to feel like heretics when they’re skeptical. They can’t win.
  - xtracto 3y ago
    One should trust science, just dont trust scientists. I was "around" science for a good chunk of my life (both mom and that used to be academics, and I spent 8 years in academia myself doing a phd and postdoc). The amount of crap studies, politics and bullshit that happens in those circles will make you realise how sad the state of the "advancement of science" is. And my experience was in 3 very different countries. We desperately need something like AI to be able to synthesize and filter scientific publications
jtaft 3y ago
With an 95% confidence interval which doesn’t include zero, doesn’t it mean that it’s statistically significant? Assuming data is valid and unbiased of course. Not a statistician, just curious.
- wxnx 3y ago
  Nope, you're thinking of regression coefficients, where you'd be correct that usually the null hypothesis is $\beta = 0$. In this case, what's being reported are odds ratios, so the null hypothesis would be that OR = 1. The parent comment's point is that although the reported effect is significant at $\alpha = 0.05$ (the usual "95% CI" you mentioned), there are other problems that render their test of this hypothesis less than valid.
  - jtaft 3y ago
    Ah thank you, had to read up on odds ratio. edit for those curious about odds ratio https://www.ncbi.nlm.nih.gov/books/NBK431098/#:~:text=The%20odds%20ratio%20is%20the,to%20be%20found%20with%20exposure https://www.ncbi.nlm.nih.gov/books/NBK431098/#:~:text=The%20....
- jaggederest 3y ago
  For odds ratio, you're looking for > 1.0, as 1.0 implies "the usual odds" i.e. the null hypothesis.
- bogeholm 3y ago
  > With an 95% confidence interval which doesn’t include zero, doesn’t it mean that it’s statistically significant? That’s explained here: https://xkcd.com/882/ https://xkcd.com/882/
danShumway 3y ago
It's honestly embarrassing to see this article somehow climb to the top of HN. It's a mess of statistical bad practice, self-reported data, and potentially confounded or ignored variables. Normally when I see a bad study there's like one or two serious problems with methodology, but when I read this through it's almost just every couple of paragraphs that the authors will say something or describe methodology that should be giving the reader pause. From literally the first paragraph in the intro: > Changes in diagnostic definitions and guidelines and increased testing availability and funding have made major contributions to this increase in diagnosed cases; under the added impacts of changes in dietary, environmental, and other exposures affecting the intrauterine environment, ASD prevalence has reached unprecedented proportions. Those two sentences contradict each other! You can't just tie them together with a semicolon like one thought implies the other. I'm not even saying that autism cases aren't actually rising at all, but you can't just go "our diagnostic criteria have changed; therefore environmental and dietary exposures are the cause." You have to actually put in the bare-bottom minimal amount of work to describe why you think that diagnostic criteria and social awareness aren't the primary causes, you can't just claim that changing diagnostic criteria itself implies diets are to blame. It's unsurprising that somebody who would write this way would do bad statistical analysis.
- bogeholm 3y ago
  >> Changes in … > Those two sentences contradict each other! Id say that it’s a non sequitur; the first part of the sentence before the semicolon states something completely different than what follows, so the ‘impacts’ can’t be ‘added’ - they don’t have the same units.
  - danShumway 3y ago
    Fair point; it's not that changing diagnostics mean that there can't be environmental/dietary changes contributing to increased autism cases, it's that the article just entirely skips over that part. It describes an outcome, describes a potential cause that could explain that outcome and then says, "and obviously also diets too."
- bornfreddy 3y ago
vegetablepotpie 3y ago
I read the article. I fell for it. And I upvoted it. I understood enough that this study found a correlation and that this was based on surveys. I thought it was an interesting finding, and concluded that this correlation should be examined more closely with more rigorous studies. I did not go into the details of methodology and statistics and did not conclude, like you did, that this study has dubious value. This is a trap that the public find themselves in with science reporting. Many people on HN have technical training to grasp these concepts but not understand them. I my self program, and use many of the same intellectual building blocks scientists use in the execution of my job. But I am not a scientist. I am not a scientist is the key point because it means that I do not understand science. I know the “process” of science. I’ve read scientific papers. I’ve done toy experiments in school and in university. But I don’t understand it. To draw an analogy, Programming is a perception of reality. There are things that I do that I can never explain to management because they do not have direct experience with it. The “identity a bird in the park” XKCD comic is a meme of this concept [1] notwithstanding advances in AI research. Like programming, science is a perception of reality. Like my management, I may have taken statistics, I know what confidence intervals are. But I have not lived the experience of building an experiment. Getting results, analyzing them, and constructing the distinctions necessary to reach an interesting and valid conclusion. If you’ve gone though that process, you know where to look for problems in a study. I and many people don’t. We will at best say, further study is needed, and at worst say that diet soda causes autism. The public depends on experts to enter the conversation and share why things are wrong. This of course gets into the problem of “lies will make it half way around the world while the truth is still putting on its shoes”. Retractions may be made, but never perceived. This makes us vulnerable to bad faith actors employing the gish gallop and there’s not a general purpose solution to that. [1] https://xkcd.com/1425/ https://xkcd.com/1425/
smsm42 3y ago
"Vaccines cause autism" is old and tired, but sodas are fashionable bad guys, so "sodas cause autism" is cool.
tomjen3 3y ago
And people wonder why we go to the comments first and the article send, if at all.
lost_tourist 3y ago
Yeah I had a nursing friend bring this up to me and I looked up the paper; i don't know jack about the medical industry but these numbers were ridiculous and the methods atrocious lol. I told her that the stats were basically useless which made the paper useless. She didn't like my answer. My two or so diet sodas a day habit will likely continue.

dkasper 3y ago

Yeah, I’m gonna need to see some replication with stronger methodology than self reporting and better confidence interval before I put much weight on this study.

xwdv 3y ago
[flagged]
- beebmam 3y ago
  Why discourage people from any behavior without evidence or good reason?
  - xwdv 3y ago
    Soda is generally very bad for one’s overall health. Although a person might not care about themselves enough to stop, perhaps they will care if their child is at risk.
- jeremyjh 3y ago
  Yes, misleading people for their own good is such a great strategy, it always works out just fine.
- deleted 3y ago
  [deleted]
- Avshalom 3y ago
  what is a harmful soda?
- achandlerwhite 3y ago
  "harmful"... citation needed
- nikau 3y ago
  > Why not put weight on Well they are drinking diet soda...
- ftxbro 3y ago
  [flagged]

ToValueFunfetti 3y ago

They're at least acknowledging obesity as a confound, but I don't see any reference to sugar sodas or caffeine in a Ctrl-F + skim.

adrr 3y ago
I didn’t see any reference to controlling for age which is has been shown to be one the biggest factors in autism.

deleted 3y ago

[deleted]

tpmx 3y ago

The r/science discussion of this paper 3 days ago is interesting: https://old.reddit.com/r/science/comments/16t4eyg/drinking_diet_sodas_and_aspartamesweetened/ https://old.reddit.com/r/science/comments/16t4eyg/drinking_d...

AdmiralAsshat 3y ago

What was it Kurt Vonnegut said? "Pity the reader." The average person has little to no understanding of statistics. The study is coming from a (seemingly) reputable university source. What reason would the uninformed have to be skeptical?

tpmx 3y ago
It's published in Nutrients. https://en.wikipedia.org/wiki/MDPI#Resignations_of_editors https://en.wikipedia.org/wiki/MDPI#Resignations_of_editors In August 2018, 10 senior editors (including the editor-in-chief) of the journal Nutrients resigned, alleging that MDPI forced the replacement of the editor-in-chief because of his high editorial standards and for resisting pressure to "accept manuscripts of mediocre quality and importance.
dimal 3y ago
Yet, on other topics, people have “trust science” drilled into them, and are made to feel like heretics when they’re skeptical. They can’t win.
- xtracto 3y ago
  One should trust science, just dont trust scientists. I was "around" science for a good chunk of my life (both mom and that used to be academics, and I spent 8 years in academia myself doing a phd and postdoc). The amount of crap studies, politics and bullshit that happens in those circles will make you realise how sad the state of the "advancement of science" is. And my experience was in 3 very different countries. We desperately need something like AI to be able to synthesize and filter scientific publications

jtaft 3y ago

With an 95% confidence interval which doesn’t include zero, doesn’t it mean that it’s statistically significant? Assuming data is valid and unbiased of course. Not a statistician, just curious.

wxnx 3y ago
Nope, you're thinking of regression coefficients, where you'd be correct that usually the null hypothesis is $\beta = 0$. In this case, what's being reported are odds ratios, so the null hypothesis would be that OR = 1. The parent comment's point is that although the reported effect is significant at $\alpha = 0.05$ (the usual "95% CI" you mentioned), there are other problems that render their test of this hypothesis less than valid.
- jtaft 3y ago
  Ah thank you, had to read up on odds ratio. edit for those curious about odds ratio https://www.ncbi.nlm.nih.gov/books/NBK431098/#:~:text=The%20odds%20ratio%20is%20the,to%20be%20found%20with%20exposure https://www.ncbi.nlm.nih.gov/books/NBK431098/#:~:text=The%20....
jaggederest 3y ago
For odds ratio, you're looking for > 1.0, as 1.0 implies "the usual odds" i.e. the null hypothesis.
bogeholm 3y ago
> With an 95% confidence interval which doesn’t include zero, doesn’t it mean that it’s statistically significant? That’s explained here: https://xkcd.com/882/ https://xkcd.com/882/

danShumway 3y ago

It's honestly embarrassing to see this article somehow climb to the top of HN. It's a mess of statistical bad practice, self-reported data, and potentially confounded or ignored variables. Normally when I see a bad study there's like one or two serious problems with methodology, but when I read this through it's almost just every couple of paragraphs that the authors will say something or describe methodology that should be giving the reader pause. From literally the first paragraph in the intro: > Changes in diagnostic definitions and guidelines and increased testing availability and funding have made major contributions to this increase in diagnosed cases; under the added impacts of changes in dietary, environmental, and other exposures affecting the intrauterine environment, ASD prevalence has reached unprecedented proportions. Those two sentences contradict each other! You can't just tie them together with a semicolon like one thought implies the other. I'm not even saying that autism cases aren't actually rising at all, but you can't just go "our diagnostic criteria have changed; therefore environmental and dietary exposures are the cause." You have to actually put in the bare-bottom minimal amount of work to describe why you think that diagnostic criteria and social awareness aren't the primary causes, you can't just claim that changing diagnostic criteria itself implies diets are to blame. It's unsurprising that somebody who would write this way would do bad statistical analysis.

bogeholm 3y ago
>> Changes in … > Those two sentences contradict each other! Id say that it’s a non sequitur; the first part of the sentence before the semicolon states something completely different than what follows, so the ‘impacts’ can’t be ‘added’ - they don’t have the same units.
- danShumway 3y ago
  Fair point; it's not that changing diagnostics mean that there can't be environmental/dietary changes contributing to increased autism cases, it's that the article just entirely skips over that part. It describes an outcome, describes a potential cause that could explain that outcome and then says, "and obviously also diets too."
bornfreddy 3y ago

vegetablepotpie 3y ago

I read the article. I fell for it. And I upvoted it. I understood enough that this study found a correlation and that this was based on surveys. I thought it was an interesting finding, and concluded that this correlation should be examined more closely with more rigorous studies. I did not go into the details of methodology and statistics and did not conclude, like you did, that this study has dubious value. This is a trap that the public find themselves in with science reporting. Many people on HN have technical training to grasp these concepts but not understand them. I my self program, and use many of the same intellectual building blocks scientists use in the execution of my job. But I am not a scientist. I am not a scientist is the key point because it means that I do not understand science. I know the “process” of science. I’ve read scientific papers. I’ve done toy experiments in school and in university. But I don’t understand it. To draw an analogy, Programming is a perception of reality. There are things that I do that I can never explain to management because they do not have direct experience with it. The “identity a bird in the park” XKCD comic is a meme of this concept [1] notwithstanding advances in AI research. Like programming, science is a perception of reality. Like my management, I may have taken statistics, I know what confidence intervals are. But I have not lived the experience of building an experiment. Getting results, analyzing them, and constructing the distinctions necessary to reach an interesting and valid conclusion. If you’ve gone though that process, you know where to look for problems in a study. I and many people don’t. We will at best say, further study is needed, and at worst say that diet soda causes autism. The public depends on experts to enter the conversation and share why things are wrong. This of course gets into the problem of “lies will make it half way around the world while the truth is still putting on its shoes”. Retractions may be made, but never perceived. This makes us vulnerable to bad faith actors employing the gish gallop and there’s not a general purpose solution to that. [1] https://xkcd.com/1425/ https://xkcd.com/1425/

smsm42 3y ago

"Vaccines cause autism" is old and tired, but sodas are fashionable bad guys, so "sodas cause autism" is cool.

tomjen3 3y ago

And people wonder why we go to the comments first and the article send, if at all.

lost_tourist 3y ago

Yeah I had a nursing friend bring this up to me and I looked up the paper; i don't know jack about the medical industry but these numbers were ridiculous and the methods atrocious lol. I told her that the stats were basically useless which made the paper useless. She didn't like my answer. My two or so diet sodas a day habit will likely continue.