4 ms·

This paper is based on garbage data: 2-4 voluntary food surveys per participant, reflecting 24-hour windows, separated by a median of 9 years from the outcome.

by timr 2y ago

This paper is based on garbage data: 2-4 voluntary food surveys per participant, reflecting 24-hour windows, separated by a median of 9 years from the outcome. Try to recall everything you ate in the last 24 hours, including quantities of calories by food group. Accurate? Now, imagine trying to correlate this with a health outcome a decade from now. On top of that, imagine that your study participants can just skip reporting anything at all if they don't feel like it. Do you think there might be a bias in the people who respond? That there might be a bias in the people who "remember" eating large quantities of plant-based foods? That there might be a bias in the people who respond to follow-up? In a just world, these kinds of nutritional epi papers wouldn't be publishable. No matter how many co-variates you put in the regression, there's simply no way to eliminate the bias.

jltsiren 2y ago
I think you are approaching this from the wrong direction. Small complex long-term effects in humans are fundamentally inaccessible to the kind of science that collects and analyzes data. Either the necessary data cannot be collected at all, or collecting it would be too expensive, time-consuming, and/or unethical to be worth it. When the truth is inaccessible, science focuses on establishing the boundaries of what can be known. You collect the data that can be collected and see what can be determined from that. Or you reuse data that already exists and try to make new justifiable conclusions from that.
- timr 2y ago
  > Small complex long-term effects in humans are fundamentally inaccessible to the kind of science that collects and analyzes data. Either the necessary data cannot be collected at all, or collecting it would be too expensive, time-consuming, and/or unethical to be worth it. When the truth is inaccessible, science focuses on establishing the boundaries of what can be known. That requires reliable data. If the only data you have is hopelessly confounded, then you don't just shrug and accept it as "truthy" because it's dressed up like science and is published in a journal, and hey, that's the best you can do anyway. Science is about the quality of the methodology you use, not the grandeur of the institution that surrounds it. If your data doesn't answer the question, then it doesn't answer the question. (For whatever it's worth, I'm biased in favor of the hypothesis, but crap experiements are crap no matter what I believe.)
  - jltsiren 2y ago
    This was not an experimental study, as the word "biobank" in the abstract should have made clear. Biobanks are about collecting more detailed data on more individuals than would be possible for any particular study, and spending a lot of effort to ensure the quality and consistency of the data. Any conclusion you can make from biobank data is a hypothesis likely worth investigating. And publishing, because further studies would likely take years. You would then collect new data designed to answer the specific question in your subsequent study. But due to financial constraints, that new data would likely be less detailed, smaller in scale, and of lower quality.

by timr 2y ago

jltsiren 2y ago

I think you are approaching this from the wrong direction. Small complex long-term effects in humans are fundamentally inaccessible to the kind of science that collects and analyzes data. Either the necessary data cannot be collected at all, or collecting it would be too expensive, time-consuming, and/or unethical to be worth it. When the truth is inaccessible, science focuses on establishing the boundaries of what can be known. You collect the data that can be collected and see what can be determined from that. Or you reuse data that already exists and try to make new justifiable conclusions from that.

timr 2y ago
> Small complex long-term effects in humans are fundamentally inaccessible to the kind of science that collects and analyzes data. Either the necessary data cannot be collected at all, or collecting it would be too expensive, time-consuming, and/or unethical to be worth it. When the truth is inaccessible, science focuses on establishing the boundaries of what can be known. That requires reliable data. If the only data you have is hopelessly confounded, then you don't just shrug and accept it as "truthy" because it's dressed up like science and is published in a journal, and hey, that's the best you can do anyway. Science is about the quality of the methodology you use, not the grandeur of the institution that surrounds it. If your data doesn't answer the question, then it doesn't answer the question. (For whatever it's worth, I'm biased in favor of the hypothesis, but crap experiements are crap no matter what I believe.)
- jltsiren 2y ago
  This was not an experimental study, as the word "biobank" in the abstract should have made clear. Biobanks are about collecting more detailed data on more individuals than would be possible for any particular study, and spending a lot of effort to ensure the quality and consistency of the data. Any conclusion you can make from biobank data is a hypothesis likely worth investigating. And publishing, because further studies would likely take years. You would then collect new data designed to answer the specific question in your subsequent study. But due to financial constraints, that new data would likely be less detailed, smaller in scale, and of lower quality.