8 ms·

Journal that published faulty black plastic study removed from science index

105 points by namanyayg 2y ago

latentcall 2y ago
I doubt this matters for the general population. My folks had already thrown out their black plastic utensils.
- deskr 2y ago
  Sadly you don't even need a mistaken research to cause damage. One nutter on youtube is enough.
lores 2y ago
Good. Not catching a simple multiplication error is a terrible indictment of their peer review process. On the other hand, peer review is doomed and getting more doomed by the day, and I expect more and more journals will fail if the quality criteria get fairly applied.
- crazygringo 2y ago
  I never thought of peer review as being meant to verify calculations. But rather to ensure the experimental methodology was sound, it cites the appropriate previous work, that it is significant enough, etc. I actually think that calculation verification and statistical significance verification is a great idea, but shouldn't be done by "peers", but rather someone who specializes only in that. More of a type of auditor.
  - burkaman 2y ago
    I don't think peer reviewers need to be redoing every calculation, but you might expect them to be familiar enough with the field that "toxins in kitchen utensils are close to the EPA limit" would be surprising enough that they'd question it and double check it.
    - bpodgursky 2y ago
      How many researchers do you think are working on cookware toxins? "Peer" is used broadly here. The article made waves because it didn't surprise anyone, because in fact a ton of household staples are full of toxins (spices are full of lead, etc).
      - pessimizer 2y ago
        > The article made waves because it didn't surprise anyone So the reason it made waves is because it confirmed everyone's previous biases. Which is why people defend it even when that "confirmation" was based on an error of simple arithmetic. Reminds me of a bunch of studies that were taken out by Excel errors, like that economics failure about national debt going higher than GDP being some sort of trigger for societal failure. The people who boosted that study were the people who were saying the same thing, with no study, for no other reason than that the numbers are of a similar magnitude. After it was shown that the study showed no such thing, its supporters insisted that it was still true because of course it was true. https://stanfordreview.org/clarifying-the-implications-of-the-reinhart-rogoff-excel-error/ https://stanfordreview.org/clarifying-the-implications-of-th...
      - burkaman 2y ago
        I don't agree that it didn't surprise anyone. Flame retardants are surprising, black plastic being worse than others is surprising, common household tools nearly at the legal toxin limit is surprising. Even the general idea of plastic being potentially bad for you is still new information for a lot of people. Your comment is the first time I've ever heard anything about lead in spices, and I read a lot of news. In my experience, very few things are actually "common knowledge", it just seems that way when everyone in our particular social circles knows something.
        Dylan16807 2y ago
        > common household tools nearly at the legal toxin limit is surprising Personally, I would expect a lot of products to be near a lot of limits for various toxins. Especially around food, my expectation is for limits to be safe with a nice-sized buffer, sometimes too big of a buffer. The point of having limits is that products inside them are fine.
- coldtea 2y ago
  >Not catching a simple multiplication error is a terrible indictment of their peer review process Most papers are full of errors nobody ever catches (or cares to catch).
  - _yb2s 2y ago
    Yes, as an academic scientist when I occasionally need to actually reproduce something from published literature to use in my own research- I almost always find at least half a dozen errors, even in a well respected journal. The only way to really find such errors is to painstakingly reproduce the work, and that is not part of the publication process- it would double the time and cost.
- wyager 2y ago
  "Peer review" has been next to useless in most fields for the last 50-100 years at least. We need a new epistemic benchmark.
- PunchTornado 2y ago
  I reviewed articles and never verified their calculations because it is not part of what a reviewer should do. A reviewer should never try and replicate the experiment or crunch numbers. It should look at the methodology, background, significance, possible enhancements etc.
- mrguyorama 2y ago
  Peer review is not an important part of the scientific method actually. It's not even necessary. There was no peer review for most of what we now take for granted in science. The Journal Nature did not make peer review standard until 1973. Science is about making reality the arbiter of truth. What is true is what you can demonstrate in an experiment pretty much anywhere. Science is self-correcting not because other scientists take a look at your work before it gets published, but because a theory that is not backed up by reality will be harder to build on, make worse predictions, and generally not be useful for actual work. It doesn't matter how much "hype" built up around LK-99 for example, because you could not build anything out of it, because it didn't work as claimed, and no amount of belief can overcome that. Science is self correcting because the universe is not convinced by rhetoric, and will only ever follow it's own rules. Tellingly, scientists who have been around for a while do not consider a published paper as especially meaningful. They do not consider peer review as vouching for a paper's correctness. They have seen hundreds of papers pass peer review and make it into Science and Nature and turn out to be bunk. No scientific process will be perfect, so some amount of papers should be expected to be wrong, no matter how genuinely careful and perfect the scientists who wrote it were. It's really sad how over-indexed the lay-public has gotten on peer review. It's just not how science works. It can still be somewhat useful for more popular journals who have to pick and choose what to publish anyway, but it's only useful if the peer reviewers are themselves domain experts in a rigorous and formalized scientific discipline.
- _yb2s 2y ago
  As an academic scientist that does peer review- I would never consider redoing math unless the conclusion seemed suspicious. Peer reviewers don’t have a lot of time and need to assume good faith and careful basic competency from the authors. We spend realistically about an hour to review a study that might have taken a whole team of authors half a decade. Mostly we are looking to confirm that the work is presented clearly, and that the evidence is sufficient to support the conclusions. Peer review is little more than proofreading from an unbiased 3rd party.
- userbinator 2y ago
  In the case of these journals, "peer" simply means "someone who shares the same agenda", and they won't care about the science or even the numerical results as long as the conclusion agrees with their ideology. The extremist envrionmentalists have been pushing their idiotic agenda for far too long, and it's good to see the sane people are now pushing back.
Workaccount2 2y ago
The study was off by a factor of 10, and the levels of flame retardants found was actually under the FDA guidelines for maximum daily exposure. However, in my own assessment, my maximum daily exposure to ingested flame retardants should be zero, rather than some off the cuff figure that was certainly done with appeasing industry in mind. So yes, I still ditched my plastic cookware in favor of stainless steel and silicone, and am still happy that the erroneous study got my attention.
- JKCalhoun 2y ago
  Agree. I was not shocked by the levels of flame retardants but that they were present at all.
  - nancyminusone 2y ago
    Why should this be shocking? 1) Make thing that operates on hot surfaces/ near fire 2) Materials thing is made of known for being flammable 3) Add extra chemicals to help stop thing from catching on fire Seems pretty logical to me.
    - Reason077 2y ago
      Because many of these flame retardants are not only environmentally toxic, but seriously incompatible with human life. They shouldn’t be present in products designed to come in contact with food, when many good alternatives are available (inherently heat-resistant thermoplastics, silicone, stainless steel, etc)
      - pfdietz 2y ago
        > but seriously incompatible with human life. There is no good evidence of this for the chemical in question.
        tannedNerd 2y ago
        [flagged]
Thorrez 2y ago
An off-by-10x multiplication error in a published paper? How about this off-by-100x multiplication error I discovered in a published paper[1]? AFAIK, that paper hasn't been retracted. I was meaning to contact the author and publisher, but never got around to it. A mitigating factor is that the error is made in regard to a side topic of the paper, not the main topic of the paper. [1] https://news.ycombinator.com/item?id=27419018 https://news.ycombinator.com/item?id=27419018
- seltzered_ 2y ago
  The article is framed in a weird way. Theres a number of journals which have been deranked recently (mdpi, 'frontiers in' https://www.timeshighereducation.com/news/finland-downgrades-mdpi-and-frontiers-will-others-follow-suit https://www.timeshighereducation.com/news/finland-downgrades... ) Also, the author said they plan to issue a correction. See https://nationalpost.com/news/canada/black-plastic https://nationalpost.com/news/canada/black-plastic "Lead author Megan Liu, science and policy manager at Toxic-Free Future, described the mistake as a “typo” and said her co-authors have submitted a correction to the journal. The error remains in the online version but Liu said she anticipates it will be updated soon. “However, it is important to note that this does not impact our results,” Liu told National Post. “The levels of flame retardants that we found in black plastic household items are still of high concern, and our recommendations remain the same.”" The correction/corrigendum note is here: https://www.sciencedirect.com/science/article/pii/S004565352402811X https://www.sciencedirect.com/science/article/pii/S004565352...
vinni2 2y ago
I trashed all my black plastic utensils and bought stainless steel ones. I can’t decide if I did the right thing or not.
- deleted 2y ago
  [deleted]
- mentalgear 2y ago
  It is. Stainless steel or glass (chemically inert) are far superior. Even 10x less flame retardants and other chemicals are still way too much. Not to mention the FDA exposure levels being too high.
  - vinni2 2y ago
    So paper was still right to warn us about it but not for the right reasons.
    - tredre3 2y ago
      The paper was right and for the right reasons. They just decided to lie to catch more headlines (nobody would have cared about it being high but under the FDA limit). It was just more effective to say "oopsie our bad but it changes nothing to our recommendations". I'm sure the authors weight the impact on their reputation versus the benefits of catching more headlines for the public good. I think it all worked as intended.
  - nancyminusone 2y ago
    Stainless steel production makes hexavalent chromium, and glass can chip and be ingested without realizing. All hazards in moderation.
    - jjulius 2y ago
      Yes, but you're leaving out some important context, context which still renders them safer than black plastic. Regarding stainless steel, that's during production of the item, not during cooking. You'd have to be welding, or using heat hot enough to cut through the steel, to release it in your own kitchen. Regarding glass, you can check for chips and the condition of your glass far better than you can for microplastics. One is also likely to leech (plastics) over the course of its life, while one is not (glass).
mentalgear 2y ago
Ah, the revenge of the mighty plastic industry pulling its strings.
- Dylan16807 2y ago
  Do you think the error is fake somehow, or do you think this is too much punishment for the number of retractions the journal has had?
deleted 2y ago
[deleted]
ajdude 2y ago
When this study was first published, Adam Ragusea did a video on it[1]. 12 minutes in, he points out that he thinks there is a basic arithmetic error in the paper. This was well before anything was on the news, and he was the only one talking about that before everything blew up. I'm surprised more people didn't catch it. [1] https://m.youtube.com/watch?v=58HUM40gDPU https://m.youtube.com/watch?v=58HUM40gDPU
- bitpush 2y ago
  Came here to say this. It's amazing how correct Adam is with his analysis.
johnea 2y ago
I never read any of these articles, but I also never use plastic cookware. Heat does leach plastisizers and other chemicals into food. Microplastics do move from cookware into food. I have a set of pots/pans and accompanying utensils, all stainless, that I've used for decades, and plan to use for the rest of my life. This also constitutes a reduction in constantly produced plastic waste. Try not to use ANY plastic...
dang 2y ago
Related (in reverse order): Huge math error corrected in black plastic study; authors say it doesn't matter - https://news.ycombinator.com/item?id=42437201 https://news.ycombinator.com/item?id=42437201 - Dec 2024 (112 comments) A simple math error sparked a panic about black plastic kitchen utensils - https://news.ycombinator.com/item?id=42400008 https://news.ycombinator.com/item?id=42400008 - Dec 2024 (176 comments) Cooking with black plastic is particularly crucial to avoid - https://news.ycombinator.com/item?id=41996156 https://news.ycombinator.com/item?id=41996156 - Oct 2024 (845 comments)
Snoozus 2y ago
The study has other problems as well. The typical customer electronics cases where you would find the flame retardants would be mostly from ABS. Kitchen utensils used in a pan can not be from ABS or they would melt. So they are made from silicone or nylon. There is a table in the paper, but its vague at best.
unparagoned 2y ago
It’s funny that this article makes a mistake in its reporting. There wasn’t an error in the levels. But just how they compared it to the guidlines

105 points by namanyayg 2y ago

latentcall 2y ago

I doubt this matters for the general population. My folks had already thrown out their black plastic utensils.

deskr 2y ago
Sadly you don't even need a mistaken research to cause damage. One nutter on youtube is enough.

lores 2y ago

Good. Not catching a simple multiplication error is a terrible indictment of their peer review process. On the other hand, peer review is doomed and getting more doomed by the day, and I expect more and more journals will fail if the quality criteria get fairly applied.

crazygringo 2y ago
I never thought of peer review as being meant to verify calculations. But rather to ensure the experimental methodology was sound, it cites the appropriate previous work, that it is significant enough, etc. I actually think that calculation verification and statistical significance verification is a great idea, but shouldn't be done by "peers", but rather someone who specializes only in that. More of a type of auditor.
- burkaman 2y ago
  I don't think peer reviewers need to be redoing every calculation, but you might expect them to be familiar enough with the field that "toxins in kitchen utensils are close to the EPA limit" would be surprising enough that they'd question it and double check it.
  - bpodgursky 2y ago
    How many researchers do you think are working on cookware toxins? "Peer" is used broadly here. The article made waves because it didn't surprise anyone, because in fact a ton of household staples are full of toxins (spices are full of lead, etc).
    - pessimizer 2y ago
      > The article made waves because it didn't surprise anyone So the reason it made waves is because it confirmed everyone's previous biases. Which is why people defend it even when that "confirmation" was based on an error of simple arithmetic. Reminds me of a bunch of studies that were taken out by Excel errors, like that economics failure about national debt going higher than GDP being some sort of trigger for societal failure. The people who boosted that study were the people who were saying the same thing, with no study, for no other reason than that the numbers are of a similar magnitude. After it was shown that the study showed no such thing, its supporters insisted that it was still true because of course it was true. https://stanfordreview.org/clarifying-the-implications-of-the-reinhart-rogoff-excel-error/ https://stanfordreview.org/clarifying-the-implications-of-th...
    - burkaman 2y ago
      I don't agree that it didn't surprise anyone. Flame retardants are surprising, black plastic being worse than others is surprising, common household tools nearly at the legal toxin limit is surprising. Even the general idea of plastic being potentially bad for you is still new information for a lot of people. Your comment is the first time I've ever heard anything about lead in spices, and I read a lot of news. In my experience, very few things are actually "common knowledge", it just seems that way when everyone in our particular social circles knows something.
      - Dylan16807 2y ago
        > common household tools nearly at the legal toxin limit is surprising Personally, I would expect a lot of products to be near a lot of limits for various toxins. Especially around food, my expectation is for limits to be safe with a nice-sized buffer, sometimes too big of a buffer. The point of having limits is that products inside them are fine.
coldtea 2y ago
>Not catching a simple multiplication error is a terrible indictment of their peer review process Most papers are full of errors nobody ever catches (or cares to catch).
- _yb2s 2y ago
  Yes, as an academic scientist when I occasionally need to actually reproduce something from published literature to use in my own research- I almost always find at least half a dozen errors, even in a well respected journal. The only way to really find such errors is to painstakingly reproduce the work, and that is not part of the publication process- it would double the time and cost.
wyager 2y ago
"Peer review" has been next to useless in most fields for the last 50-100 years at least. We need a new epistemic benchmark.
PunchTornado 2y ago
I reviewed articles and never verified their calculations because it is not part of what a reviewer should do. A reviewer should never try and replicate the experiment or crunch numbers. It should look at the methodology, background, significance, possible enhancements etc.
mrguyorama 2y ago
Peer review is not an important part of the scientific method actually. It's not even necessary. There was no peer review for most of what we now take for granted in science. The Journal Nature did not make peer review standard until 1973. Science is about making reality the arbiter of truth. What is true is what you can demonstrate in an experiment pretty much anywhere. Science is self-correcting not because other scientists take a look at your work before it gets published, but because a theory that is not backed up by reality will be harder to build on, make worse predictions, and generally not be useful for actual work. It doesn't matter how much "hype" built up around LK-99 for example, because you could not build anything out of it, because it didn't work as claimed, and no amount of belief can overcome that. Science is self correcting because the universe is not convinced by rhetoric, and will only ever follow it's own rules. Tellingly, scientists who have been around for a while do not consider a published paper as especially meaningful. They do not consider peer review as vouching for a paper's correctness. They have seen hundreds of papers pass peer review and make it into Science and Nature and turn out to be bunk. No scientific process will be perfect, so some amount of papers should be expected to be wrong, no matter how genuinely careful and perfect the scientists who wrote it were. It's really sad how over-indexed the lay-public has gotten on peer review. It's just not how science works. It can still be somewhat useful for more popular journals who have to pick and choose what to publish anyway, but it's only useful if the peer reviewers are themselves domain experts in a rigorous and formalized scientific discipline.
_yb2s 2y ago
As an academic scientist that does peer review- I would never consider redoing math unless the conclusion seemed suspicious. Peer reviewers don’t have a lot of time and need to assume good faith and careful basic competency from the authors. We spend realistically about an hour to review a study that might have taken a whole team of authors half a decade. Mostly we are looking to confirm that the work is presented clearly, and that the evidence is sufficient to support the conclusions. Peer review is little more than proofreading from an unbiased 3rd party.
userbinator 2y ago
In the case of these journals, "peer" simply means "someone who shares the same agenda", and they won't care about the science or even the numerical results as long as the conclusion agrees with their ideology. The extremist envrionmentalists have been pushing their idiotic agenda for far too long, and it's good to see the sane people are now pushing back.

Workaccount2 2y ago

The study was off by a factor of 10, and the levels of flame retardants found was actually under the FDA guidelines for maximum daily exposure. However, in my own assessment, my maximum daily exposure to ingested flame retardants should be zero, rather than some off the cuff figure that was certainly done with appeasing industry in mind. So yes, I still ditched my plastic cookware in favor of stainless steel and silicone, and am still happy that the erroneous study got my attention.

JKCalhoun 2y ago
Agree. I was not shocked by the levels of flame retardants but that they were present at all.
- nancyminusone 2y ago
  Why should this be shocking? 1) Make thing that operates on hot surfaces/ near fire 2) Materials thing is made of known for being flammable 3) Add extra chemicals to help stop thing from catching on fire Seems pretty logical to me.
  - Reason077 2y ago
    Because many of these flame retardants are not only environmentally toxic, but seriously incompatible with human life. They shouldn’t be present in products designed to come in contact with food, when many good alternatives are available (inherently heat-resistant thermoplastics, silicone, stainless steel, etc)
    - pfdietz 2y ago
      > but seriously incompatible with human life. There is no good evidence of this for the chemical in question.
      - tannedNerd 2y ago
        [flagged]

Thorrez 2y ago

An off-by-10x multiplication error in a published paper? How about this off-by-100x multiplication error I discovered in a published paper[1]? AFAIK, that paper hasn't been retracted. I was meaning to contact the author and publisher, but never got around to it. A mitigating factor is that the error is made in regard to a side topic of the paper, not the main topic of the paper. [1] https://news.ycombinator.com/item?id=27419018 https://news.ycombinator.com/item?id=27419018

seltzered_ 2y ago
The article is framed in a weird way. Theres a number of journals which have been deranked recently (mdpi, 'frontiers in' https://www.timeshighereducation.com/news/finland-downgrades-mdpi-and-frontiers-will-others-follow-suit https://www.timeshighereducation.com/news/finland-downgrades... ) Also, the author said they plan to issue a correction. See https://nationalpost.com/news/canada/black-plastic https://nationalpost.com/news/canada/black-plastic "Lead author Megan Liu, science and policy manager at Toxic-Free Future, described the mistake as a “typo” and said her co-authors have submitted a correction to the journal. The error remains in the online version but Liu said she anticipates it will be updated soon. “However, it is important to note that this does not impact our results,” Liu told National Post. “The levels of flame retardants that we found in black plastic household items are still of high concern, and our recommendations remain the same.”" The correction/corrigendum note is here: https://www.sciencedirect.com/science/article/pii/S004565352402811X https://www.sciencedirect.com/science/article/pii/S004565352...

vinni2 2y ago

I trashed all my black plastic utensils and bought stainless steel ones. I can’t decide if I did the right thing or not.

deleted 2y ago
[deleted]
mentalgear 2y ago
It is. Stainless steel or glass (chemically inert) are far superior. Even 10x less flame retardants and other chemicals are still way too much. Not to mention the FDA exposure levels being too high.
- vinni2 2y ago
  So paper was still right to warn us about it but not for the right reasons.
  - tredre3 2y ago
    The paper was right and for the right reasons. They just decided to lie to catch more headlines (nobody would have cared about it being high but under the FDA limit). It was just more effective to say "oopsie our bad but it changes nothing to our recommendations". I'm sure the authors weight the impact on their reputation versus the benefits of catching more headlines for the public good. I think it all worked as intended.
- nancyminusone 2y ago
  Stainless steel production makes hexavalent chromium, and glass can chip and be ingested without realizing. All hazards in moderation.
  - jjulius 2y ago
    Yes, but you're leaving out some important context, context which still renders them safer than black plastic. Regarding stainless steel, that's during production of the item, not during cooking. You'd have to be welding, or using heat hot enough to cut through the steel, to release it in your own kitchen. Regarding glass, you can check for chips and the condition of your glass far better than you can for microplastics. One is also likely to leech (plastics) over the course of its life, while one is not (glass).

mentalgear 2y ago

Ah, the revenge of the mighty plastic industry pulling its strings.

Dylan16807 2y ago
Do you think the error is fake somehow, or do you think this is too much punishment for the number of retractions the journal has had?

deleted 2y ago

[deleted]

ajdude 2y ago

When this study was first published, Adam Ragusea did a video on it[1]. 12 minutes in, he points out that he thinks there is a basic arithmetic error in the paper. This was well before anything was on the news, and he was the only one talking about that before everything blew up. I'm surprised more people didn't catch it. [1] https://m.youtube.com/watch?v=58HUM40gDPU https://m.youtube.com/watch?v=58HUM40gDPU

bitpush 2y ago
Came here to say this. It's amazing how correct Adam is with his analysis.

johnea 2y ago

I never read any of these articles, but I also never use plastic cookware. Heat does leach plastisizers and other chemicals into food. Microplastics do move from cookware into food. I have a set of pots/pans and accompanying utensils, all stainless, that I've used for decades, and plan to use for the rest of my life. This also constitutes a reduction in constantly produced plastic waste. Try not to use ANY plastic...

dang 2y ago

Related (in reverse order): Huge math error corrected in black plastic study; authors say it doesn't matter - https://news.ycombinator.com/item?id=42437201 https://news.ycombinator.com/item?id=42437201 - Dec 2024 (112 comments) A simple math error sparked a panic about black plastic kitchen utensils - https://news.ycombinator.com/item?id=42400008 https://news.ycombinator.com/item?id=42400008 - Dec 2024 (176 comments) Cooking with black plastic is particularly crucial to avoid - https://news.ycombinator.com/item?id=41996156 https://news.ycombinator.com/item?id=41996156 - Oct 2024 (845 comments)

Snoozus 2y ago

The study has other problems as well. The typical customer electronics cases where you would find the flame retardants would be mostly from ABS. Kitchen utensils used in a pan can not be from ABS or they would melt. So they are made from silicone or nylon. There is a table in the paper, but its vague at best.

unparagoned 2y ago

It’s funny that this article makes a mistake in its reporting. There wasn’t an error in the levels. But just how they compared it to the guidlines