20 ms·

Note, all of these studies use a low standard of evidence (food questionnaires and the like), and they generate low signal (barely registering positive/negative

Note, all of these studies use a low standard of evidence (food questionnaires and the like), and they generate low signal (barely registering positive/negative). Read John Ioannidis @ Stanford for why these methods should banned, and why nutritional research currently produces mostly irrelevant data. Harvard, unfortunately, has doubled-down on this style of science. It's not helpful that the press supports these methods, without skeptical commentary.

rmtech 7y ago
Got a reference for ioannidis on this topic specifically?
- jeremyw 7y ago
  Sure, start with these. Good references, too. Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How https://academic.oup.com/advances/article/9/4/367/5055939 https://academic.oup.com/advances/article/9/4/367/5055939 Implausible results in human nutrition research https://www.bmj.com/content/347/bmj.f6698 https://www.bmj.com/content/347/bmj.f6698 His most famous paper, Why Most Published Research Findings Are False https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... Here's a talk on nutrition science methods, along with a specific audience moment with Walter Willet, former head of Harvard Nutrition dept, and advocate for his questionnaires. https://www.youtube.com/watch?v=KTAbx4i8Dyg https://www.youtube.com/watch?v=KTAbx4i8Dyg -- https://youtu.be/KTAbx4i8Dyg?t=23m40s https://youtu.be/KTAbx4i8Dyg?t=23m40s
  - o09rdk 7y ago
    After reading the BMJ piece I wondered if any study strategy would satisfy Ioannidis, to be honest. I came away from it feeling like his calls to eliminate observational studies and smaller randomized trials are misleading in that he seems to believe that all realistic studies should be eliminated. He seems to be in favor of large randomized-controlled comprehensive diet protocols, for example, where many aspects of diets are manipulated simultaneously. He dismisses them too, though, for very good reasons. Given that there were additional reasons to be skeptical of such studies he didn't mention, I was left wondering if his pleas for study designs should be heeded very much (as opposed to his criticism of specific flaws). I love his work, but am wondering if it's best approached as worst-case criticism rather than an inspiration for what to do.
    - jeremyw 7y ago
      That's interesting. I read him (in total) as being quite optimistic about the way forward, if intertia and self-interest can be overcome. He's listed as an author in a number of nutritional and other studies. One might see those as leaning in his direction, if he's willing to be mentioned. You might contact him if you have additional doubts.
jfim 7y ago
> Note, all of these studies use a low standard of evidence (food questionnaires and the like) Sounds like someone is optimizing for the number of papers written per grant dollar, as opposed to generating quality science. Why do expensive research when you can just print a few questionnaires, get a few underpaid grad students to administer them, then trawl the data for acceptable p-values for publication?
gyuserbti 7y ago
I have a lot of respect for Ioannadis but also find it reprehensible to talk about banning or otherwise forbidding large classes of research. Science only progresses when unfettered; prohibitions never help. If a study is flawed, address the flaws. More information might not help, but it never hurts. These flawed studies exist for a reason -- ideal studies are usually the stuff of fantasy.
- Talyen42 7y ago
  The current system is absolutely prohibiting good science from being done, through incentive structures on the faculty tenure/review process weighing heavily towards frequency/quantity of published material. faculty review processes are mostly new to the last 10-20 years ("accountability" suddenly became a thing in academia), as the quantity of published material has gone up by orders of magnitude more than the number of researchers.
trappist 7y ago
I see this as an epistemological failure in nutrition that plagues other fields as well. We have all this knowledge of human physiology and metabolism, and make very little use of it in nutrition, acting as though we can't draw logical conclusions from what we know about those things, or trust those conclusions, or at least use them to apply serious skepticism to the apparent implications of low-quality data. For myself, until you can tell me by what means a particular food produces the claimed outcome, I dismiss your epidemiological nutrition study.
- andrewf 7y ago
  We have all this knowledge of human physiology and metabolism I think the depth of knowledge is very uneven. We're not sure how Tylenol works. (Disclaimer: I work in software, not bioscience)
  - nzzzz 7y ago
    There will never be explanations on "how paracetamol (the principal substance of tylenol) works" because.. it doesn't work. It doesn't fare better than placebo. https://www.evidentlycochrane.net/paracetamol-widely-used-ineffective/ https://www.evidentlycochrane.net/paracetamol-widely-used-in... At least, placebos don't harm the user. There's evidence paracetamol is harmful. https://ard.bmj.com/content/75/3/552 https://ard.bmj.com/content/75/3/552 And dosages higher than 4g/day are known to destroy the liver. Paracetamol overdose is one of the most painful way to kill yourself. Its abuse being freely sold over the counter makes it the number one cause of liver failure in the US and UK. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1403265/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1403265/ >Paracetamol’s toxicity is also the single biggest cause of acute liver failure in the United States. Cases have been rising for six years, according to a study published in December (Hepatology 2005;42:1364-72). By 2003, the drug accounted for just over half the cases of acute liver failure, and about half of these cases were the result of unintentional overdose. It boggles the mind the substance has not been put under harsher control yet.
    - circlefavshape 7y ago

rmtech 7y ago

Got a reference for ioannidis on this topic specifically?

jeremyw 7y ago
Sure, start with these. Good references, too. Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How https://academic.oup.com/advances/article/9/4/367/5055939 https://academic.oup.com/advances/article/9/4/367/5055939 Implausible results in human nutrition research https://www.bmj.com/content/347/bmj.f6698 https://www.bmj.com/content/347/bmj.f6698 His most famous paper, Why Most Published Research Findings Are False https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... Here's a talk on nutrition science methods, along with a specific audience moment with Walter Willet, former head of Harvard Nutrition dept, and advocate for his questionnaires. https://www.youtube.com/watch?v=KTAbx4i8Dyg https://www.youtube.com/watch?v=KTAbx4i8Dyg -- https://youtu.be/KTAbx4i8Dyg?t=23m40s https://youtu.be/KTAbx4i8Dyg?t=23m40s
- o09rdk 7y ago
  After reading the BMJ piece I wondered if any study strategy would satisfy Ioannidis, to be honest. I came away from it feeling like his calls to eliminate observational studies and smaller randomized trials are misleading in that he seems to believe that all realistic studies should be eliminated. He seems to be in favor of large randomized-controlled comprehensive diet protocols, for example, where many aspects of diets are manipulated simultaneously. He dismisses them too, though, for very good reasons. Given that there were additional reasons to be skeptical of such studies he didn't mention, I was left wondering if his pleas for study designs should be heeded very much (as opposed to his criticism of specific flaws). I love his work, but am wondering if it's best approached as worst-case criticism rather than an inspiration for what to do.
  - jeremyw 7y ago
    That's interesting. I read him (in total) as being quite optimistic about the way forward, if intertia and self-interest can be overcome. He's listed as an author in a number of nutritional and other studies. One might see those as leaning in his direction, if he's willing to be mentioned. You might contact him if you have additional doubts.

jfim 7y ago

> Note, all of these studies use a low standard of evidence (food questionnaires and the like) Sounds like someone is optimizing for the number of papers written per grant dollar, as opposed to generating quality science. Why do expensive research when you can just print a few questionnaires, get a few underpaid grad students to administer them, then trawl the data for acceptable p-values for publication?

gyuserbti 7y ago

I have a lot of respect for Ioannadis but also find it reprehensible to talk about banning or otherwise forbidding large classes of research. Science only progresses when unfettered; prohibitions never help. If a study is flawed, address the flaws. More information might not help, but it never hurts. These flawed studies exist for a reason -- ideal studies are usually the stuff of fantasy.

Talyen42 7y ago
The current system is absolutely prohibiting good science from being done, through incentive structures on the faculty tenure/review process weighing heavily towards frequency/quantity of published material. faculty review processes are mostly new to the last 10-20 years ("accountability" suddenly became a thing in academia), as the quantity of published material has gone up by orders of magnitude more than the number of researchers.

trappist 7y ago

I see this as an epistemological failure in nutrition that plagues other fields as well. We have all this knowledge of human physiology and metabolism, and make very little use of it in nutrition, acting as though we can't draw logical conclusions from what we know about those things, or trust those conclusions, or at least use them to apply serious skepticism to the apparent implications of low-quality data. For myself, until you can tell me by what means a particular food produces the claimed outcome, I dismiss your epidemiological nutrition study.

andrewf 7y ago
We have all this knowledge of human physiology and metabolism I think the depth of knowledge is very uneven. We're not sure how Tylenol works. (Disclaimer: I work in software, not bioscience)
- nzzzz 7y ago
  There will never be explanations on "how paracetamol (the principal substance of tylenol) works" because.. it doesn't work. It doesn't fare better than placebo. https://www.evidentlycochrane.net/paracetamol-widely-used-ineffective/ https://www.evidentlycochrane.net/paracetamol-widely-used-in... At least, placebos don't harm the user. There's evidence paracetamol is harmful. https://ard.bmj.com/content/75/3/552 https://ard.bmj.com/content/75/3/552 And dosages higher than 4g/day are known to destroy the liver. Paracetamol overdose is one of the most painful way to kill yourself. Its abuse being freely sold over the counter makes it the number one cause of liver failure in the US and UK. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1403265/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1403265/ >Paracetamol’s toxicity is also the single biggest cause of acute liver failure in the United States. Cases have been rising for six years, according to a study published in December (Hepatology 2005;42:1364-72). By 2003, the drug accounted for just over half the cases of acute liver failure, and about half of these cases were the result of unintentional overdose. It boggles the mind the substance has not been put under harsher control yet.
  - circlefavshape 7y ago