5 ms·

Your analysis seems to portray all scientists as pure hearted. May I remind you of the latest Stanford scandal where the president of Stanford was found to have

by dmbche 3y ago

Your analysis seems to portray all scientists as pure hearted. May I remind you of the latest Stanford scandal where the president of Stanford was found to have manipulated data? Today, publications do not serve the same purpose as they did before the internet. It is trivial today to write a convincing paper without research and getting that published(www.theatlantic.com/ideas/archive/2018/10/new-sokal-hoax/572212/&sa=U&ved=2ahUKEwjnp5mRtsiAAxVwF1kFHesBDC8QFnoECAkQAg&usg=AOvVaw0t_Bo31BrT5D9zHBdmNAqi).

matthewdgreen 3y ago
No subset of humanity is “pure hearted.” Fraud and malice will exist in everything people do. Fortunately these fraudulent incidents seem relatively rare, when one compares the number of reported incidents to the number of publications and scientists. But this doesn’t change anything. The benefit of scientific publication is to make it easier to detect and verify incorrect results, which is exactly what happened in this case. I understand that it’s frustrating it didn’t happen instantly. And I also understand that it’s deeply frustrating that some undeserving person accumulated status points with non-scientists based on fraud, and that let them take a high-status position outside of their field. (I think maybe you should assign some blame to the Stanford Trustees for this, but that’s up to you.) None of this means we’d be better off making publication more difficult: it means the metrics are bad. PS When a TFA raises something like “the replication crisis” and then entangles it with accusations of deliberate fraud (high profile but exceedingly rare) it’s like trying to have a serious conversation about automobile accidents, but spending half the conversation on a handful of rare incidents of intentional vehicular homicide. You’re not going to get useful solutions out of this conversation, because it’s (perhaps deliberately) misunderstanding the impact and causes of the problem.
- dmbche 3y ago
  For your analogy on car accidents - a notable difference between both is that in the case of car accidents, we are able to get numbers on when, how and why they happen and then make conclusions from that. In this case, we are not even aware of most events of fraud/"bad papers"/manipulation - the "crisis" is that we are losing faith in the science we are doing - results that were cornerstones of entire fields are found to be nonreproducible, making all the work built on top of it pointless.(psychology, cancer, economics, etc - I'm being very broad) At this point, we don't know how deep the rot goes. We are at the point of recognizing that it's real, and looking for solutions. For car accidents, we're past that - we're just arguing about what are the best solutions. For the replication crisis, we're trying to find a way forward. Like that scene in The Thing, where they test the blood? We're at the point where we don't know who to trust. Ps: what's a tfa?
  - deleted 3y ago
    [deleted]
- mike_hearn 3y ago
  Fraud isn't exceedingly rare :( It only seems that way because academia doesn't pay anyone to find it, reacts to volunteer reports by ignoring it, and the media generally isn't interested. Fraud is so frequent and easy to find that there are volunteers who in their spare time manage to routinely uncover not just individual instances of fraud but entire companies whose sole purpose is to generate and sell fake papers on an industrial scale. https://www.nature.com/articles/d41586-023-01780-w https://www.nature.com/articles/d41586-023-01780-w Fraud is so easy and common that there are a steady stream of journals which publish entire editions consisting of nothing but AI generated articles! https://www.nature.com/articles/d41586-021-03035-y https://www.nature.com/articles/d41586-021-03035-y Despite being written as a joke over a decade ago, you can page through an endless stream of papers that were generated by SciGen - a Perl script - and yet they are getting published: https://pubpeer.com/search?q=scigen https://pubpeer.com/search?q=scigen The problem is so prevalent that some people created the Problematic Paper Screener, a tool that automatically locates articles that contain text indicative of auto-generation. https://dbrech.irit.fr/pls/apex/f?p=9999:1 https://dbrech.irit.fr/pls/apex/f?p=9999:1:::::: This is all pre-ChatGPT, and is just the researchers who can't be bothered writing a paper at all. The more serious problem is all the human written fraudulent papers with bad data and bad methodologies that are never detected, or only detected by randos with blogs or Twitter accounts that you never hear around.
  - dmbche 3y ago
    Thanks you - just discovered Scigen, these links are incredible
  - matthewdgreen 3y ago
    The wonderful thing about the western world is that most countries value freedom of the press. The dark side of this is that you can spin up your own “scientific journal” and charge people to publish in it, game the rankings like any common SEO scam, and nobody will stop you because (especially here in the US) you’re exercising your first amendment rights. Then people can fill it with nonsense and even script-generated fake papers. People outside the scientific community can also scam more “legitimate” for-profit journals in various ways, resulting in more silly publications that the actual scientific community has to filter out. It’s very annoying. None of this has any more bearing on fraud by professional scientists than, say, the existence of some garbage-filled Wikimedia server or a badly-edited Wikipedia page means that the Wikipedia editors themselves are fraudsters.

by dmbche 3y ago

matthewdgreen 3y ago

No subset of humanity is “pure hearted.” Fraud and malice will exist in everything people do. Fortunately these fraudulent incidents seem relatively rare, when one compares the number of reported incidents to the number of publications and scientists. But this doesn’t change anything. The benefit of scientific publication is to make it easier to detect and verify incorrect results, which is exactly what happened in this case. I understand that it’s frustrating it didn’t happen instantly. And I also understand that it’s deeply frustrating that some undeserving person accumulated status points with non-scientists based on fraud, and that let them take a high-status position outside of their field. (I think maybe you should assign some blame to the Stanford Trustees for this, but that’s up to you.) None of this means we’d be better off making publication more difficult: it means the metrics are bad. PS When a TFA raises something like “the replication crisis” and then entangles it with accusations of deliberate fraud (high profile but exceedingly rare) it’s like trying to have a serious conversation about automobile accidents, but spending half the conversation on a handful of rare incidents of intentional vehicular homicide. You’re not going to get useful solutions out of this conversation, because it’s (perhaps deliberately) misunderstanding the impact and causes of the problem.

dmbche 3y ago
For your analogy on car accidents - a notable difference between both is that in the case of car accidents, we are able to get numbers on when, how and why they happen and then make conclusions from that. In this case, we are not even aware of most events of fraud/"bad papers"/manipulation - the "crisis" is that we are losing faith in the science we are doing - results that were cornerstones of entire fields are found to be nonreproducible, making all the work built on top of it pointless.(psychology, cancer, economics, etc - I'm being very broad) At this point, we don't know how deep the rot goes. We are at the point of recognizing that it's real, and looking for solutions. For car accidents, we're past that - we're just arguing about what are the best solutions. For the replication crisis, we're trying to find a way forward. Like that scene in The Thing, where they test the blood? We're at the point where we don't know who to trust. Ps: what's a tfa?
- deleted 3y ago
  [deleted]
mike_hearn 3y ago
Fraud isn't exceedingly rare :( It only seems that way because academia doesn't pay anyone to find it, reacts to volunteer reports by ignoring it, and the media generally isn't interested. Fraud is so frequent and easy to find that there are volunteers who in their spare time manage to routinely uncover not just individual instances of fraud but entire companies whose sole purpose is to generate and sell fake papers on an industrial scale. https://www.nature.com/articles/d41586-023-01780-w https://www.nature.com/articles/d41586-023-01780-w Fraud is so easy and common that there are a steady stream of journals which publish entire editions consisting of nothing but AI generated articles! https://www.nature.com/articles/d41586-021-03035-y https://www.nature.com/articles/d41586-021-03035-y Despite being written as a joke over a decade ago, you can page through an endless stream of papers that were generated by SciGen - a Perl script - and yet they are getting published: https://pubpeer.com/search?q=scigen https://pubpeer.com/search?q=scigen The problem is so prevalent that some people created the Problematic Paper Screener, a tool that automatically locates articles that contain text indicative of auto-generation. https://dbrech.irit.fr/pls/apex/f?p=9999:1 https://dbrech.irit.fr/pls/apex/f?p=9999:1:::::: This is all pre-ChatGPT, and is just the researchers who can't be bothered writing a paper at all. The more serious problem is all the human written fraudulent papers with bad data and bad methodologies that are never detected, or only detected by randos with blogs or Twitter accounts that you never hear around.
- dmbche 3y ago
  Thanks you - just discovered Scigen, these links are incredible
- matthewdgreen 3y ago
  The wonderful thing about the western world is that most countries value freedom of the press. The dark side of this is that you can spin up your own “scientific journal” and charge people to publish in it, game the rankings like any common SEO scam, and nobody will stop you because (especially here in the US) you’re exercising your first amendment rights. Then people can fill it with nonsense and even script-generated fake papers. People outside the scientific community can also scam more “legitimate” for-profit journals in various ways, resulting in more silly publications that the actual scientific community has to filter out. It’s very annoying. None of this has any more bearing on fraud by professional scientists than, say, the existence of some garbage-filled Wikimedia server or a badly-edited Wikipedia page means that the Wikipedia editors themselves are fraudsters.