36 ms·

'Serious errors' in research linking deaths to red meat

190 points by rgrieselhuber 4y ago

Mountain_Skies 4y ago
As the replication crisis grows, or we're simply becoming more aware of it, I do wonder how much of this is due to poorly aligned incentives causing researchers to error on the side of what will get the most attention versus researchers in general simply being sloppy. Much discussion has been had on what to do if it is the former but if it is the latter, what needs to change in the education process so studies are better designed to end up with errors so frequently?
- cratermoon 4y ago
  In this case I'm pretty sure that the research in question was about the same quality as the work produced by Tobacco Industry Research Committee or the Sugar Research Foundation.
- pvaldes 4y ago
  > what needs to change in the education process so studies are better designed Remove politics, free kidnapped and gagged science.
  - seventytwo 4y ago
    Politics is an emergent property of human interaction. It cannot be removed.
  - dapf 4y ago
    > Remove politics That needs the remotion of taxation. I agree.
  - karaterobot 4y ago
    I think that's a separate issue that would not have affected this particular case.
  - xhkkffbf 4y ago
    That's just hard to do with a topic like this. There are committed people on both sides of the issue who know what they want to hear. That's not a good environment for science to find a neutral answer.
  - mumblemumble 4y ago
    Politics and education are certainly factors, but I would argue that they are minor. The bigger issue is the natural incentive structures that academia has created. If you want to have a career, you get both attention and funding in return for headline-grabbing results. This creates two huge second-order effects: first, everyone's incentivized to look for big impressive new effects. Second, focusing on confirmatory work is a great way to tank your career. And those create a third order effect: people who cut corners thrive, and there's not much to keep that in check because people who cautiously vet their work and others' get stuck in adjunct professor hell until they finally give up and exit the field. Politics just adds some spin onto that. Education, I just don't think that's even a possible solution. The math involved in some of these fancier statistical techniques can be devilishly hard to get right, and it's often simply asking too much to expect someone to be an expert at both that and molecular biology. You could argue that that means people should look for competent co-authors to cover that side of things. But, see above - current incentive structures don't really favor the researcher who says, "I know! Let's bring on some nerd whose only job is to find reasons why we shouldn't publish!"
    - pvaldes 4y ago
      If we assign grant points to people that flood the precious journal space with fake articles claiming over-hyped promises, maybe we should give grant points also to the people able to spot the lies on those kind of articles. After all they are serving to the advance of science also (and saving thousands of expensive hours of research to other). If you want to promote it, just create incentives. Otherwise, If the cheater wins in a 95% of the times and the verifier does not have anything to gain (and will damage their own research by the time invested on this) after a while only cheaters remain [1]. At middle term, all research tend towards a sort of expensive shamanism in well illuminated white rooms. [1] (And of course the non cheaters will be actively blocked from this point).
      - MPSimmons 4y ago
        Like a bug bounty for academic publications?
- colechristensen 4y ago
  I think it is not enough of a sense of quality and too much pressure or gamification of getting things published. And fundamentally lots of researchers need to be more interested in finding the truth and considerably better at statistics.
- dale_glass 4y ago
  Individual studies shouldn't be the primary focus. It shouldn't fall to every individual researcher to be fully accurate and honest. This is just not possible because people are imperfect, and have all the wrong incentives in many cases. What we need is a system designed to deal with this better. My suggestion would be greater incentives to the publication of negative results, and incentives to reproduce research. Create a system that encourages reproduction as a second step to peer review. This should also increase quality. If your research can't be easily reproduced it's probably not doing a whole lot of good even if ultimately you got the right answers. Test driven research, if you will.
  - jonhohle 4y ago
    Or even requirements for replication amongst groups with separate funding sources. If everyone needs replication for publishing, the incentive to replicate other’s work increases.
  - Tao3300 4y ago
    > It shouldn't fall to every individual researcher to be fully... honest. Really? I don't know, "Don't be a fraud" might be a good Rule No. 1
    - fennecfoxen 4y ago
      It’s more that a system should not allow a single fraud to shape its understanding and its policy. Assume fraud exists.
    - naasking 4y ago
      Fraud is only a small part of the problem. No open data, poorly described methodology so reproductions sometimes aren't even possible, too little funding for reproductions, p-hacking, no incentives to publish negative results, journals incentivized to publish sensational results to increase their impact factor even though sensational results are likely false thus perpetuating misinformation. These are the main drivers.
  - FrenchAmerican 4y ago
- dc-programmer 4y ago
  Personally this is one of the reasons I chose to go into software instead of statistics when I was a young math major. The idea that you could make a small mistake that invalidates all your work and you might not even know it until much later or even never always made me uncomfortable. With software I can write tests till my neuroticism has been placated
  - jefftk 4y ago
    If all the programmers who write tests go into industry and none of them go into academia, then the state of software in academia will be... oh, I see, the status quo.
    - pca006132 4y ago
      No, they do write tests, just 1. Include the ones that passed and discard those that failed. 2. Run the tests several times and report only the best result. 3. Keep some of the details secret (data, some weird parameters, or maybe the whole repository), tweak something, run again, get some nice figures and publish a new paper.
      - WalterBright 4y ago
        Caltech has Millikan's (oil drop experiment demonstrating quantization of charge) notebooks. He repeated the experiment until the data fit his theory. Fortunately, he turned out to be right.
- jwond 4y ago
  > I do wonder how much of this is due to poorly aligned incentives causing researchers to error on the side of what will get the most attention versus researchers in general simply being sloppy. I think we should also consider the possibility that some of it might be ideologically motivated. For example someone who thinks people shouldn’t eat red meat due to climate change, animal rights, etc. (and who possibly belongs to a social circle where such beliefs are popular) might be motivated to fabricate results that could convince some people to eat less red meat (hypothetically, not saying that is the case here).
  - deleted 4y ago
    [deleted]
- yboris 4y ago
  "Poorly aligned incentives" in the nutritional sciences is, as far as I can tell, mostly various food lobbies getting their product to look good. The reason there is "confusion" about whether many animal products are good or bad is because the corporations making billions of dollars off the product fund biased research. Consider looking at the research digests from a non-profit (that sells no products, has no ads, etc) focusing on improving your health: https://nutritionfacts.org/ https://nutritionfacts.org/
  - netmonk 4y ago
    How a non-profit pays for hosting website cost, writing report, and so on and so on… being a non-profit doesn’t mean you are tied to corporations and free of any conflict of interest. If you start looking carefully you will see that most non-profit org aren’t unbiased and honest. And since long time… For instance this whole vegan trend is based on some radical religious opinion to control fertility and lower class reproduction by promoting a hormonal dysfonctionnal diet. All those from “official scientific paper” written by those religious fanatic few decades ago.
    - yboris 4y ago
      > vegan trend is based on some radical religious opinion What a bold empirical claim. Could you share any evidence of this?
- contravariant 4y ago
  It has probably been a mistake to have so many articles that are effectively "We measured A and its value suggests the following theory" rather than "To test theory B we measured A and found it did(n't) agree" I reckon that lack of a good theory is the by far best predictor for non-replicable results.
  - pca006132 4y ago
    I think there are several issues with this. In these fields, theories (not so fundamental ones) may not be very useful, in contrast with fields like Mathematics or Physics where a hypothesis may lead to a lot of other theoretical results. For example, if material A has property X, there will be a lot of applications for it, but this hypothesis cannot be used as a foundation for other theoretical research. As a result, it is not useful to just state a hypothesis: you have to verify it as well, and negative results are not really valued because it means nothing to others if you rejected something they don't care about.
    - contravariant 4y ago
      Not sure if properties of materials is the best example, those tend to have some theory behind them that, if not explains, at least defines the property and its consequences. If one research team measures the stiffness of a material it's easy enough to make a test that replicates their result. Medical research in particular seems to suffer a lot from papers where something has been measured with no supporting theory whatsoever and consequently no real information other than "If you do what we did you probably get the same result". Rather unsurprisingly this is hard to replicate because there's no theory telling you what is and isn't relevant to replicate. I'm not saying this is easy to fix, but maybe we should focus more effort on papers that expand the theory, rather than papers that at best show that something might be the case but we don't know why.
      - watwut 4y ago
        It makes sense to me to first collect small amounts of data, only then formulate theories and only then verify those with larger experiments. Especially when experiments are expensive. It does not make sense to act like small studies are proof of anything the way people like to use those studies on the internet. It is wrong that researchers are effectively punished for reproduction, verification etc. But the existence of small data collection studies on itself is not wrong.
- derbOac 4y ago
  You're framing this as if they're mutually exclusive options, which they're not. My impression is that it's "all of the above": the incentive structure in academics is completely broken, and this incentivizes both unethical practices, as well as being sloppy "to get things done". I can think of numerous examples of both from my career, from working with various colleagues and in stories I've heard. All of this is the tip of the iceberg I think: what you hear about usually are cases of blatant fraud, with high-profile or high-importance research. The bulk of greyer malpractice, with softer squishier levels of ethical dubiousness, and more indirect, cumulative impacts on peoples' careers and literature trajectories, are more hidden. It's gotten to the point where I no longer put much stock in the reputation of individual researchers. Too many times I know in the best case there's an institution, or department, or subcommunity, or doctoral students or postdocs behind it, with who knows what sort of manipulation or luck. In the worst case there's years of fraud or other forms of corruption. Call me inappropriately cynical but I didn't start out this way.
narrator 4y ago
I think any study that says eating meat is unhealthy has to have a much higher standard of proof because there is so much money floating around from environmental and animal rights activists to fund any scientist who is willing to make that case.
- gpt5 4y ago
  You could make the same argument about a study that says that meat is good.
  - GenerocUsername 4y ago
    Except that eating meat is a common activity globally for all of human history... It would be like publishing a paper that says walking upright is harmful and then people start crab walking. Like yes you can explain many back problems with walking upright, but does that actually justify a different form of walking?
    - z9znz 4y ago
      > eating meat is a common activity globally for all of human history... That's a very bold stroke, especially since most of human history is unwritten and not well understood. This topic is "red" meat. That does not include fish, insects, and other non-red meats. The human diet was historically highly dependent upon whatever was locally available until the last 100 years (or less). Also, the gut biomes of humans were historically very different than ours are today. Now we have easy availability of lower quality meat (typical modern factory farmed meat which we consume larger quantities of), very different gut biomes, more sedentary lifestyles, and any number of other "new" things that make us abnormal humans in human history spectrum; so it's fair to believe that we could have some negative health effects from eating red meat.
      - colordrops 4y ago
        Also the quantity and dietary proportion of meat consumed has changed significantly. People may have eaten meat long ago, but it wasn't the staple nor a daily part of the meal for most people.
andreareina 4y ago
https://archive.ph/20220831193814/https://www.thetimes.co.uk/article/serious-errors-in-research-linking-deaths-to-red-meat-qtdh28fv5 https://archive.ph/20220831193814/https://www.thetimes.co.uk...
bmacho 4y ago
[flagged]
- denton-scratch 4y ago
  Someone posted an archive.ph link up-thread. [Edit] That was a bit unhelpful. If you hit a paywall, do this: - Copy the url you're visiting to the clipboard. - Visit archive.ph - Paste the URL from your clipboard, and submit the search.
- macintux 4y ago
  No. https://news.ycombinator.com/item?id=32462320 https://news.ycombinator.com/item?id=32462320
deleted 4y ago
[deleted]
TrispusAttucks 4y ago
When science becomes driven by political agendas it ceases to be science.
- gruez 4y ago
  What's the political agenda behind eating more/less red meat?
  - YetAnotherNick 4y ago
    Climate change
    - nerdponx 4y ago
      ...is not a political agenda.
      - GaveDrohl 4y ago
        If you're claiming that no politicians have made this their agenda, then I guess the planet is in trouble!
      - YetAnotherNick 4y ago
        But it is, even if climate change is true. e.g. the original paper aim has nothing to do with climate change, but it mentions climate change 6 times in the paper and 5 more in the reference.
- TheRealPomax 4y ago
  No, it doesn't. That's the kind of attitude that ignores perfectly good science that proves something you don't like just because you didn't like the authors or what they stand for. What drives the science is irrelevant in the question of whether it's still science. As long as the data and methodology are sound, science was done. It's when either or both are compromised in order to make false claims that something ceases to be science.
gwerbret 4y ago
Of the lead author of this critical letter, the article says: "In her initial correspondence with The Lancet, Stanton declared that she was a part-time employee of Devenish Nutrition, an agricultural-technology and animal-feed company in which she also holds shares."
- rgrieselhuber 4y ago
  It is a conspiracy theory to associate commercial connections with scientific research and publications.
  - davikr 4y ago
    It certainly is a conflict of interest which should be stated.
    - kspacewalk2 4y ago
      It was stated: >In her initial correspondence with The Lancet, Stanton declared that she was a part-time employee of Devenish Nutrition, an agricultural-technology and animal-feed company in which she also holds shares. >One of the co-authors of the open letter, Christopher Elliott, of Queen’s, declared grants to his research institute from Devenish.
  - colechristensen 4y ago
    It's called a conflict of interest, and being worried about such conflicts is not paranoia, it is perfectly reasonable care. It adds quite a bit of doubt when the "truth" uncovered is particularly advantageous to the cause of the conflict.
    - rgrieselhuber 4y ago
      I agree, my comment was sarcasm because this principle is not universally applied.
      - 4y ago
endisneigh 4y ago
more diet silliness. look, this is the ultimate diet: - Eat food. Not too much. Mostly plants. Mostly unprocessed. Nutrition science has come up with so many contradictory conclusions in the past decades.
- Silverback_VII 4y ago
  Why not start eating humans ? this would solve many issues at the same time. More nutritious and ecological than eating plants I suppose.
  - jefftk 4y ago
    The world death rate is about 8 per 1000 people per year. If there are 50 pounds of edible meat on a body, then human deaths could supply 0.001 pounds per living person. This is also low-quality gamy meat because the average age of death among people is incredibly high for meat animals. Enacting worldwide cannibalism for humans would provide as much additional meat as spending one tenth of a cent per person annually on conventional low-end meat. Not worth the social debate or the medical risk.
    - Oarch 4y ago
      I enjoyed reading this way too much.
    - DiggyJohnson 4y ago
      Just watched S4E1 of Always Sunny, "Mac & Dennis: Manhunters", which was a lot of fun. Thank you for supplying this supplementary information.
  - swarnie 4y ago
    Since the median age of death is 82 (78 if free), riddled with an entire pharmacy of modern medicine i suspect that's a very, very unhealthy proposition. May as well eat roadkill.
  - Shorel 4y ago
    A modest proposal
deleted 4y ago
[deleted]
m0llusk 4y ago
This is more that a flawed study, but a serious failing of comprehension. The assertion in more detail is that IGF-1 in meat causes illness at a predictable rate, but exercise causes large spikes in IGF-1 while being associated with a drop in cancer risk. Clearly the correlation between IGF-1 is not as simple as this study asserts.
- valec 4y ago
  IGF-1 is generally bad for longevity. Exercise is beneficial in spite of spiking IGF-1, not because of
mikeatwork 4y ago
drno123 4y ago
Every PhD student should, in order to obtain PhD, not only do his original research but reproduce another published result. This will make PhD harder to get, so PhD’s will become more valuable (and more vetted), and we will have people verify published studies.
- metacritic12 4y ago
  Great idea! Reproducing is also a great way to understand the field, and can actually help you a lot with your own research and techniques. This also ensures the most famous results are the most replicated. Hugely valuable to society since the most famous results also drive the most thinking / decisions / policies.
- eganist 4y ago
  Wow, I can't think of anything practically wrong with this. I can think of plenty of reasons why it won't happen (notably the downward pressure on the candidate pipeline, which for a variety of reasons isn't an outcome research institutions don't want), but I'm wondering if the best place to apply the pressure would be the accreditation institutions and the Department of Education.
  - MonkeyMalarky 4y ago
    Might want a restriction like research from an unaffiliated institution / research group. I don't think reproducing results from someone in the same lab has much value.
    - eganist 4y ago
      Ah, that's a good idea. And it pushes a more adversarial dynamic that might motivate people to debunk eyebrow-raising conclusions from competing institutions.
  - watwut 4y ago
    One problem is that PhDs are already overworked and effectively taken advantage of. So, something would need to be removed from stuff they already do.
jimrandomh 4y ago
Primary source: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2822%2900311-7/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6... .
- jimrandomh 4y ago
  The whole thing reads like lawyers talking about bugs in a spreasheet, without being willing to show the spreadsheet itself. And everything in the spreadsheet itself is survey data, not the results of experiments, which puts it on shaky footing to begin with. My own perspective is that this entire branch of research--trying to figure out risk factors for nutrition based on non-intervential studies--is a failed paradigm which cannot reasonably be expected to produce true conclusions, has a long history of producing false conclusions that were later overturned. It's basically pseudoscience.
aaaaaaaaaaab 4y ago
Joke's on them. I never stopped eating red meat :)
andy_ppp 4y ago
The classic problem with self reporting food questionnaires is recall of the correct information. If anyone here knows accurately how many cups of ribs they had in the last year, (unless zero) you might as well just invent a random number. I have no idea how you would even follow someone's diet closely enough to be even 50% accurate and for how long? Until someone is dead. Nobody is doing accurate research on this because it would cost too much and consequently the "science" that is done is meaningless. Peter Attia makes some interesting suggestions on how to improve things (from David Allison's work) here [1]. [1] https://peterattiamd.com/nutritional-epidemiology-abolition-vs-defending-the-status-quo/ https://peterattiamd.com/nutritional-epidemiology-abolition-...
- cryptonector 4y ago
  I recently met several 'carnivores' who eat one to two pounds of meat every day, and only meat, dairy, fruit, honey, and water -- no other carbs, no sugar, no rice, no pasta, no bread, no wheat, no corn, no veggies(!) (apparently a lot of vegetables have something called "oxalates" and that's bad, though I know nothing about the topic), no alcohol, no nuts, no seeds, no seed oils, etc. They swear by it. I'd imagine that carnivores can probably estimate their red meat consumption very accurately, though I didn't ask the ones I met recently -- they were pretty sure of how much meat they eat each day, so I assume it's trivial to x365 to get "how much beef they had in the last year". Well, maybe not that trivial because some of them also have pork and chicken? https://www.google.com/search?q=carnivore+groups https://www.google.com/search?q=carnivore+groups
  - jokethrowaway 4y ago
    Carnivores are a fairly recent phenomenon. The studies linking meat to health issues were done on the McDonald's and coke population. There is also zero commercial interest in pushing meat over carb filled diets. Look inside any supermarket. It's literally 95% non-meat processed food with long expiry date, an easy manufacturing process and the added benefit of keeping people addicted and hungry.
jiggywiggy 4y ago
This is more than a replication error. Large-scale studies, with results that have too many variables, have strong tendency to underwrite the consensus. That wouldn't be such an issue, if they have huge impact on policy making. In this case the zeitgeist is against meat, for obvious reasons, but there are many more of them.
yessirwhatever 4y ago
Shocker... That's what you get when you politicize climate.
benjaminmaccini 4y ago
@dang, can we link the actual journal article[0] instead of the embellished advert to eat more red meat? Also, generally, diet is political in nature and I don't see its place here on HN. Had this article focused on something else, like the replication crisis as it pertains to nutrition research (like some of the comments seem to think), instead of promoting specific dietary views, I could see a healthy discussion starting. However, as it stands, there is nothing in the comments that is related to the article and productive. This seems to be a common trend, I can email you a list of links, if you're interested. --- For my own edification; some thoughts after reading the journal, the comments and some supplementary links. I'm trying not to put on a tinfoil hat and I could be missing the point on some of these. - The author performs funded research by and holds shares in an agri-tech company with a focus on poultry and swine[1]. This isn't an immediate conflict of interest in my mind (food professionals are commonly hired by agribusinesses), but just glancing at her previous publications[2], it's hard to find her arguing against livestock consumption in any way. - What exactly are the 'serious errors' mentioned, specific to red meat consumption? The article's primary concerns seems to be that the GBD 2019 risk factor analysis[3] a) Changes the TMREL (theoretical minimum risk exposure level) for risks to be 0 (this isn't just for red meat) b) Categorizes red meat as a risk factor in the first place c) Fails to match up with other systematic reviews put up by other research organizations. One organization mentioned, NutriRECS[4], has substantial ties to the meat industry[5] - The article fails to mention the fact that GBD groups "dietary factors" into one when linking to death, AND that there was decrease in diet-related burdens[6]. I don't see anywhere that the GDP study says red meat definitively equals more death. It's mentioned as an increased risk factor, but not elaborated on. Additionally, I don't know where this article's authors got their "36-fold" increase from. Could someone help me find that? - I'm confused more about what a quality diet should be after reading this piece. So, it seems as though the article is successful. [0] https://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2822%2900311-7/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6... [1] https://web.archive.org/web/20220710204451/https://us.devenishnutrition.com/ https://web.archive.org/web/20220710204451/https://us.deveni... [2] https://www.rcsi.com/people/profile/astanton https://www.rcsi.com/people/profile/astanton [3] https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(20)30752-2.pdf https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736... [4] https://www.nutrirecs.com https://www.nutrirecs.com [5] https://www.washingtonpost.com/business/2020/01/07/youll-never-guess-who-helped-fund-that-controversial-keep-eating-red-meat-study/ https://www.washingtonpost.com/business/2020/01/07/youll-nev... [6] Go to [3] section titled "Substantial changes compared with GBD 2017" (or just Ctrl-F for "red meat").
robertlagrant 4y ago
Never mind red meat deaths. Studies on kidney stones have been found to often involve p-forcing.
- RappingBoomer 4y ago
  what do you mean by that?
mbrodersen 4y ago
The more I learn about nutritional science the less I trust it. The amount of corruption, incompetency, and political cherry picking routinely demonstrated by nutritional “scientists” is jaw dropping.
taylodl 4y ago
This is how science is supposed to work. There really doesn't appear to be anything to see here.

190 points by rgrieselhuber 4y ago

As the replication crisis grows, or we're simply becoming more aware of it, I do wonder how much of this is due to poorly aligned incentives causing researchers to error on the side of what will get the most attention versus researchers in general simply being sloppy. Much discussion has been had on what to do if it is the former but if it is the latter, what needs to change in the education process so studies are better designed to end up with errors so frequently?

cratermoon 4y ago
In this case I'm pretty sure that the research in question was about the same quality as the work produced by Tobacco Industry Research Committee or the Sugar Research Foundation.
pvaldes 4y ago
> what needs to change in the education process so studies are better designed Remove politics, free kidnapped and gagged science.
- seventytwo 4y ago
  Politics is an emergent property of human interaction. It cannot be removed.
- dapf 4y ago
  > Remove politics That needs the remotion of taxation. I agree.
- karaterobot 4y ago
  I think that's a separate issue that would not have affected this particular case.
- xhkkffbf 4y ago
  That's just hard to do with a topic like this. There are committed people on both sides of the issue who know what they want to hear. That's not a good environment for science to find a neutral answer.
- mumblemumble 4y ago
  Politics and education are certainly factors, but I would argue that they are minor. The bigger issue is the natural incentive structures that academia has created. If you want to have a career, you get both attention and funding in return for headline-grabbing results. This creates two huge second-order effects: first, everyone's incentivized to look for big impressive new effects. Second, focusing on confirmatory work is a great way to tank your career. And those create a third order effect: people who cut corners thrive, and there's not much to keep that in check because people who cautiously vet their work and others' get stuck in adjunct professor hell until they finally give up and exit the field. Politics just adds some spin onto that. Education, I just don't think that's even a possible solution. The math involved in some of these fancier statistical techniques can be devilishly hard to get right, and it's often simply asking too much to expect someone to be an expert at both that and molecular biology. You could argue that that means people should look for competent co-authors to cover that side of things. But, see above - current incentive structures don't really favor the researcher who says, "I know! Let's bring on some nerd whose only job is to find reasons why we shouldn't publish!"
  - pvaldes 4y ago
    If we assign grant points to people that flood the precious journal space with fake articles claiming over-hyped promises, maybe we should give grant points also to the people able to spot the lies on those kind of articles. After all they are serving to the advance of science also (and saving thousands of expensive hours of research to other). If you want to promote it, just create incentives. Otherwise, If the cheater wins in a 95% of the times and the verifier does not have anything to gain (and will damage their own research by the time invested on this) after a while only cheaters remain [1]. At middle term, all research tend towards a sort of expensive shamanism in well illuminated white rooms. [1] (And of course the non cheaters will be actively blocked from this point).
    - MPSimmons 4y ago
      Like a bug bounty for academic publications?
colechristensen 4y ago
I think it is not enough of a sense of quality and too much pressure or gamification of getting things published. And fundamentally lots of researchers need to be more interested in finding the truth and considerably better at statistics.
dale_glass 4y ago
Individual studies shouldn't be the primary focus. It shouldn't fall to every individual researcher to be fully accurate and honest. This is just not possible because people are imperfect, and have all the wrong incentives in many cases. What we need is a system designed to deal with this better. My suggestion would be greater incentives to the publication of negative results, and incentives to reproduce research. Create a system that encourages reproduction as a second step to peer review. This should also increase quality. If your research can't be easily reproduced it's probably not doing a whole lot of good even if ultimately you got the right answers. Test driven research, if you will.
- jonhohle 4y ago
  Or even requirements for replication amongst groups with separate funding sources. If everyone needs replication for publishing, the incentive to replicate other’s work increases.
- Tao3300 4y ago
  > It shouldn't fall to every individual researcher to be fully... honest. Really? I don't know, "Don't be a fraud" might be a good Rule No. 1
  - fennecfoxen 4y ago
    It’s more that a system should not allow a single fraud to shape its understanding and its policy. Assume fraud exists.
  - naasking 4y ago
    Fraud is only a small part of the problem. No open data, poorly described methodology so reproductions sometimes aren't even possible, too little funding for reproductions, p-hacking, no incentives to publish negative results, journals incentivized to publish sensational results to increase their impact factor even though sensational results are likely false thus perpetuating misinformation. These are the main drivers.
- FrenchAmerican 4y ago
dc-programmer 4y ago
Personally this is one of the reasons I chose to go into software instead of statistics when I was a young math major. The idea that you could make a small mistake that invalidates all your work and you might not even know it until much later or even never always made me uncomfortable. With software I can write tests till my neuroticism has been placated
- jefftk 4y ago
  If all the programmers who write tests go into industry and none of them go into academia, then the state of software in academia will be... oh, I see, the status quo.
  - pca006132 4y ago
    No, they do write tests, just 1. Include the ones that passed and discard those that failed. 2. Run the tests several times and report only the best result. 3. Keep some of the details secret (data, some weird parameters, or maybe the whole repository), tweak something, run again, get some nice figures and publish a new paper.
    - WalterBright 4y ago
      Caltech has Millikan's (oil drop experiment demonstrating quantization of charge) notebooks. He repeated the experiment until the data fit his theory. Fortunately, he turned out to be right.
jwond 4y ago
> I do wonder how much of this is due to poorly aligned incentives causing researchers to error on the side of what will get the most attention versus researchers in general simply being sloppy. I think we should also consider the possibility that some of it might be ideologically motivated. For example someone who thinks people shouldn’t eat red meat due to climate change, animal rights, etc. (and who possibly belongs to a social circle where such beliefs are popular) might be motivated to fabricate results that could convince some people to eat less red meat (hypothetically, not saying that is the case here).
- deleted 4y ago
  [deleted]
yboris 4y ago
"Poorly aligned incentives" in the nutritional sciences is, as far as I can tell, mostly various food lobbies getting their product to look good. The reason there is "confusion" about whether many animal products are good or bad is because the corporations making billions of dollars off the product fund biased research. Consider looking at the research digests from a non-profit (that sells no products, has no ads, etc) focusing on improving your health: https://nutritionfacts.org/ https://nutritionfacts.org/
- netmonk 4y ago
  How a non-profit pays for hosting website cost, writing report, and so on and so on… being a non-profit doesn’t mean you are tied to corporations and free of any conflict of interest. If you start looking carefully you will see that most non-profit org aren’t unbiased and honest. And since long time… For instance this whole vegan trend is based on some radical religious opinion to control fertility and lower class reproduction by promoting a hormonal dysfonctionnal diet. All those from “official scientific paper” written by those religious fanatic few decades ago.
  - yboris 4y ago
    > vegan trend is based on some radical religious opinion What a bold empirical claim. Could you share any evidence of this?
contravariant 4y ago
It has probably been a mistake to have so many articles that are effectively "We measured A and its value suggests the following theory" rather than "To test theory B we measured A and found it did(n't) agree" I reckon that lack of a good theory is the by far best predictor for non-replicable results.
- pca006132 4y ago
  I think there are several issues with this. In these fields, theories (not so fundamental ones) may not be very useful, in contrast with fields like Mathematics or Physics where a hypothesis may lead to a lot of other theoretical results. For example, if material A has property X, there will be a lot of applications for it, but this hypothesis cannot be used as a foundation for other theoretical research. As a result, it is not useful to just state a hypothesis: you have to verify it as well, and negative results are not really valued because it means nothing to others if you rejected something they don't care about.
  - contravariant 4y ago
    Not sure if properties of materials is the best example, those tend to have some theory behind them that, if not explains, at least defines the property and its consequences. If one research team measures the stiffness of a material it's easy enough to make a test that replicates their result. Medical research in particular seems to suffer a lot from papers where something has been measured with no supporting theory whatsoever and consequently no real information other than "If you do what we did you probably get the same result". Rather unsurprisingly this is hard to replicate because there's no theory telling you what is and isn't relevant to replicate. I'm not saying this is easy to fix, but maybe we should focus more effort on papers that expand the theory, rather than papers that at best show that something might be the case but we don't know why.
    - watwut 4y ago
      It makes sense to me to first collect small amounts of data, only then formulate theories and only then verify those with larger experiments. Especially when experiments are expensive. It does not make sense to act like small studies are proof of anything the way people like to use those studies on the internet. It is wrong that researchers are effectively punished for reproduction, verification etc. But the existence of small data collection studies on itself is not wrong.
derbOac 4y ago
You're framing this as if they're mutually exclusive options, which they're not. My impression is that it's "all of the above": the incentive structure in academics is completely broken, and this incentivizes both unethical practices, as well as being sloppy "to get things done". I can think of numerous examples of both from my career, from working with various colleagues and in stories I've heard. All of this is the tip of the iceberg I think: what you hear about usually are cases of blatant fraud, with high-profile or high-importance research. The bulk of greyer malpractice, with softer squishier levels of ethical dubiousness, and more indirect, cumulative impacts on peoples' careers and literature trajectories, are more hidden. It's gotten to the point where I no longer put much stock in the reputation of individual researchers. Too many times I know in the best case there's an institution, or department, or subcommunity, or doctoral students or postdocs behind it, with who knows what sort of manipulation or luck. In the worst case there's years of fraud or other forms of corruption. Call me inappropriately cynical but I didn't start out this way.

narrator 4y ago

I think any study that says eating meat is unhealthy has to have a much higher standard of proof because there is so much money floating around from environmental and animal rights activists to fund any scientist who is willing to make that case.

gpt5 4y ago
You could make the same argument about a study that says that meat is good.
- GenerocUsername 4y ago
  Except that eating meat is a common activity globally for all of human history... It would be like publishing a paper that says walking upright is harmful and then people start crab walking. Like yes you can explain many back problems with walking upright, but does that actually justify a different form of walking?
  - z9znz 4y ago
    > eating meat is a common activity globally for all of human history... That's a very bold stroke, especially since most of human history is unwritten and not well understood. This topic is "red" meat. That does not include fish, insects, and other non-red meats. The human diet was historically highly dependent upon whatever was locally available until the last 100 years (or less). Also, the gut biomes of humans were historically very different than ours are today. Now we have easy availability of lower quality meat (typical modern factory farmed meat which we consume larger quantities of), very different gut biomes, more sedentary lifestyles, and any number of other "new" things that make us abnormal humans in human history spectrum; so it's fair to believe that we could have some negative health effects from eating red meat.
    - colordrops 4y ago
      Also the quantity and dietary proportion of meat consumed has changed significantly. People may have eaten meat long ago, but it wasn't the staple nor a daily part of the meal for most people.

andreareina 4y ago

https://archive.ph/20220831193814/https://www.thetimes.co.uk/article/serious-errors-in-research-linking-deaths-to-red-meat-qtdh28fv5 https://archive.ph/20220831193814/https://www.thetimes.co.uk...

bmacho 4y ago

[flagged]

denton-scratch 4y ago
Someone posted an archive.ph link up-thread. [Edit] That was a bit unhelpful. If you hit a paywall, do this: - Copy the url you're visiting to the clipboard. - Visit archive.ph - Paste the URL from your clipboard, and submit the search.
macintux 4y ago
No. https://news.ycombinator.com/item?id=32462320 https://news.ycombinator.com/item?id=32462320

deleted 4y ago

[deleted]

TrispusAttucks 4y ago

When science becomes driven by political agendas it ceases to be science.

gruez 4y ago
What's the political agenda behind eating more/less red meat?
- YetAnotherNick 4y ago
  Climate change
  - nerdponx 4y ago
    ...is not a political agenda.
    - GaveDrohl 4y ago
      If you're claiming that no politicians have made this their agenda, then I guess the planet is in trouble!
    - YetAnotherNick 4y ago
      But it is, even if climate change is true. e.g. the original paper aim has nothing to do with climate change, but it mentions climate change 6 times in the paper and 5 more in the reference.
TheRealPomax 4y ago
No, it doesn't. That's the kind of attitude that ignores perfectly good science that proves something you don't like just because you didn't like the authors or what they stand for. What drives the science is irrelevant in the question of whether it's still science. As long as the data and methodology are sound, science was done. It's when either or both are compromised in order to make false claims that something ceases to be science.

gwerbret 4y ago

Of the lead author of this critical letter, the article says: "In her initial correspondence with The Lancet, Stanton declared that she was a part-time employee of Devenish Nutrition, an agricultural-technology and animal-feed company in which she also holds shares."

rgrieselhuber 4y ago
It is a conspiracy theory to associate commercial connections with scientific research and publications.
- davikr 4y ago
  It certainly is a conflict of interest which should be stated.
  - kspacewalk2 4y ago
    It was stated: >In her initial correspondence with The Lancet, Stanton declared that she was a part-time employee of Devenish Nutrition, an agricultural-technology and animal-feed company in which she also holds shares. >One of the co-authors of the open letter, Christopher Elliott, of Queen’s, declared grants to his research institute from Devenish.
- colechristensen 4y ago
  It's called a conflict of interest, and being worried about such conflicts is not paranoia, it is perfectly reasonable care. It adds quite a bit of doubt when the "truth" uncovered is particularly advantageous to the cause of the conflict.
  - rgrieselhuber 4y ago
    I agree, my comment was sarcasm because this principle is not universally applied.
    - 4y ago

endisneigh 4y ago

more diet silliness. look, this is the ultimate diet: - Eat food. Not too much. Mostly plants. Mostly unprocessed. Nutrition science has come up with so many contradictory conclusions in the past decades.

Silverback_VII 4y ago
Why not start eating humans ? this would solve many issues at the same time. More nutritious and ecological than eating plants I suppose.
- jefftk 4y ago
  The world death rate is about 8 per 1000 people per year. If there are 50 pounds of edible meat on a body, then human deaths could supply 0.001 pounds per living person. This is also low-quality gamy meat because the average age of death among people is incredibly high for meat animals. Enacting worldwide cannibalism for humans would provide as much additional meat as spending one tenth of a cent per person annually on conventional low-end meat. Not worth the social debate or the medical risk.
  - Oarch 4y ago
    I enjoyed reading this way too much.
  - DiggyJohnson 4y ago
    Just watched S4E1 of Always Sunny, "Mac & Dennis: Manhunters", which was a lot of fun. Thank you for supplying this supplementary information.
- swarnie 4y ago
  Since the median age of death is 82 (78 if free), riddled with an entire pharmacy of modern medicine i suspect that's a very, very unhealthy proposition. May as well eat roadkill.
- Shorel 4y ago
  A modest proposal

deleted 4y ago

[deleted]

m0llusk 4y ago

This is more that a flawed study, but a serious failing of comprehension. The assertion in more detail is that IGF-1 in meat causes illness at a predictable rate, but exercise causes large spikes in IGF-1 while being associated with a drop in cancer risk. Clearly the correlation between IGF-1 is not as simple as this study asserts.

valec 4y ago
IGF-1 is generally bad for longevity. Exercise is beneficial in spite of spiking IGF-1, not because of

mikeatwork 4y ago

drno123 4y ago

Every PhD student should, in order to obtain PhD, not only do his original research but reproduce another published result. This will make PhD harder to get, so PhD’s will become more valuable (and more vetted), and we will have people verify published studies.

metacritic12 4y ago
Great idea! Reproducing is also a great way to understand the field, and can actually help you a lot with your own research and techniques. This also ensures the most famous results are the most replicated. Hugely valuable to society since the most famous results also drive the most thinking / decisions / policies.
eganist 4y ago
Wow, I can't think of anything practically wrong with this. I can think of plenty of reasons why it won't happen (notably the downward pressure on the candidate pipeline, which for a variety of reasons isn't an outcome research institutions don't want), but I'm wondering if the best place to apply the pressure would be the accreditation institutions and the Department of Education.
- MonkeyMalarky 4y ago
  Might want a restriction like research from an unaffiliated institution / research group. I don't think reproducing results from someone in the same lab has much value.
  - eganist 4y ago
    Ah, that's a good idea. And it pushes a more adversarial dynamic that might motivate people to debunk eyebrow-raising conclusions from competing institutions.
- watwut 4y ago
  One problem is that PhDs are already overworked and effectively taken advantage of. So, something would need to be removed from stuff they already do.

jimrandomh 4y ago

Primary source: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2822%2900311-7/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6... .

jimrandomh 4y ago
The whole thing reads like lawyers talking about bugs in a spreasheet, without being willing to show the spreadsheet itself. And everything in the spreadsheet itself is survey data, not the results of experiments, which puts it on shaky footing to begin with. My own perspective is that this entire branch of research--trying to figure out risk factors for nutrition based on non-intervential studies--is a failed paradigm which cannot reasonably be expected to produce true conclusions, has a long history of producing false conclusions that were later overturned. It's basically pseudoscience.

aaaaaaaaaaab 4y ago

Joke's on them. I never stopped eating red meat :)

andy_ppp 4y ago

The classic problem with self reporting food questionnaires is recall of the correct information. If anyone here knows accurately how many cups of ribs they had in the last year, (unless zero) you might as well just invent a random number. I have no idea how you would even follow someone's diet closely enough to be even 50% accurate and for how long? Until someone is dead. Nobody is doing accurate research on this because it would cost too much and consequently the "science" that is done is meaningless. Peter Attia makes some interesting suggestions on how to improve things (from David Allison's work) here [1]. [1] https://peterattiamd.com/nutritional-epidemiology-abolition-vs-defending-the-status-quo/ https://peterattiamd.com/nutritional-epidemiology-abolition-...

cryptonector 4y ago
I recently met several 'carnivores' who eat one to two pounds of meat every day, and only meat, dairy, fruit, honey, and water -- no other carbs, no sugar, no rice, no pasta, no bread, no wheat, no corn, no veggies(!) (apparently a lot of vegetables have something called "oxalates" and that's bad, though I know nothing about the topic), no alcohol, no nuts, no seeds, no seed oils, etc. They swear by it. I'd imagine that carnivores can probably estimate their red meat consumption very accurately, though I didn't ask the ones I met recently -- they were pretty sure of how much meat they eat each day, so I assume it's trivial to x365 to get "how much beef they had in the last year". Well, maybe not that trivial because some of them also have pork and chicken? https://www.google.com/search?q=carnivore+groups https://www.google.com/search?q=carnivore+groups
- jokethrowaway 4y ago
  Carnivores are a fairly recent phenomenon. The studies linking meat to health issues were done on the McDonald's and coke population. There is also zero commercial interest in pushing meat over carb filled diets. Look inside any supermarket. It's literally 95% non-meat processed food with long expiry date, an easy manufacturing process and the added benefit of keeping people addicted and hungry.

jiggywiggy 4y ago

This is more than a replication error. Large-scale studies, with results that have too many variables, have strong tendency to underwrite the consensus. That wouldn't be such an issue, if they have huge impact on policy making. In this case the zeitgeist is against meat, for obvious reasons, but there are many more of them.

yessirwhatever 4y ago

Shocker... That's what you get when you politicize climate.

benjaminmaccini 4y ago

@dang, can we link the actual journal article[0] instead of the embellished advert to eat more red meat? Also, generally, diet is political in nature and I don't see its place here on HN. Had this article focused on something else, like the replication crisis as it pertains to nutrition research (like some of the comments seem to think), instead of promoting specific dietary views, I could see a healthy discussion starting. However, as it stands, there is nothing in the comments that is related to the article and productive. This seems to be a common trend, I can email you a list of links, if you're interested. --- For my own edification; some thoughts after reading the journal, the comments and some supplementary links. I'm trying not to put on a tinfoil hat and I could be missing the point on some of these. - The author performs funded research by and holds shares in an agri-tech company with a focus on poultry and swine[1]. This isn't an immediate conflict of interest in my mind (food professionals are commonly hired by agribusinesses), but just glancing at her previous publications[2], it's hard to find her arguing against livestock consumption in any way. - What exactly are the 'serious errors' mentioned, specific to red meat consumption? The article's primary concerns seems to be that the GBD 2019 risk factor analysis[3] a) Changes the TMREL (theoretical minimum risk exposure level) for risks to be 0 (this isn't just for red meat) b) Categorizes red meat as a risk factor in the first place c) Fails to match up with other systematic reviews put up by other research organizations. One organization mentioned, NutriRECS[4], has substantial ties to the meat industry[5] - The article fails to mention the fact that GBD groups "dietary factors" into one when linking to death, AND that there was decrease in diet-related burdens[6]. I don't see anywhere that the GDP study says red meat definitively equals more death. It's mentioned as an increased risk factor, but not elaborated on. Additionally, I don't know where this article's authors got their "36-fold" increase from. Could someone help me find that? - I'm confused more about what a quality diet should be after reading this piece. So, it seems as though the article is successful. [0] https://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2822%2900311-7/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6... [1] https://web.archive.org/web/20220710204451/https://us.devenishnutrition.com/ https://web.archive.org/web/20220710204451/https://us.deveni... [2] https://www.rcsi.com/people/profile/astanton https://www.rcsi.com/people/profile/astanton [3] https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(20)30752-2.pdf https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736... [4] https://www.nutrirecs.com https://www.nutrirecs.com [5] https://www.washingtonpost.com/business/2020/01/07/youll-never-guess-who-helped-fund-that-controversial-keep-eating-red-meat-study/ https://www.washingtonpost.com/business/2020/01/07/youll-nev... [6] Go to [3] section titled "Substantial changes compared with GBD 2017" (or just Ctrl-F for "red meat").

robertlagrant 4y ago

Never mind red meat deaths. Studies on kidney stones have been found to often involve p-forcing.

RappingBoomer 4y ago
what do you mean by that?

mbrodersen 4y ago

The more I learn about nutritional science the less I trust it. The amount of corruption, incompetency, and political cherry picking routinely demonstrated by nutritional “scientists” is jaw dropping.

taylodl 4y ago

This is how science is supposed to work. There really doesn't appear to be anything to see here.