4 ms·

I saw the picture get posted on social media earlier today, and I took one look at it and thought "this looks like something from the Cursed AI Facebook group I

I saw the picture get posted on social media earlier today, and I took one look at it and thought "this looks like something from the Cursed AI Facebook group I'm in" before I read the post accompanying it and saw them saying it was published in a peer-reviewed journal. How could this possibly pass peer review?

blibble 3y ago
> How could this possibly pass peer review? they've probably outsourced that to generative AI too
- amyjess 3y ago
  I wouldn't be surprised. I've seen people say that instead of reading articles now, they just open up ChatGPT and say "Summarize the important parts of the following article for me". My experience is that ChatGPT is laughably wrong when it comes to generating any kind of factual information. At best, it's good for feeding ridiculous prompts into it and trying to see what the most cursed thing you can get to spit out is so you can have a laugh.
  - geoelectric 3y ago
    My experience is querying an LLM usually does ok if the only knowledge it needs is directly included in the prompt (including for summarization). It’s not perfect but its reliability is within the realm of human error and so isn’t significantly worse than me skimming the text myself. But it’s really unreliable if it’s leveraging “knowledge” from its training. Asking what something in an article means, for example, is pretty risky. Open-ended questions that cause knowledge dumps are very risky—the longer the answer, the riskier. Even text translation is risky, even though it’s ostensibly a simple transformation, because it has to draw the word mappings from training. And god help you if you ask it a leading question without realizing it. It’ll often just parrot your assumption back to you. To your point, though, using an LLM as an oracle/teacher would be a lot less dangerous if it were always laughably wrong but it’s usually not. It’s usually 90% right, and that’s what makes it so insidious. Picking out the 10% that isn’t is really hard, especially if you don’t know enough to second-guess it. The wrong stuff usually looks plausible and it’s surrounded by enough valid info that what little you do know seems to corroborate the rest.
- gbjw 3y ago
  The reviewers' names are listed on the publisher's (Frontier) site [0, top right] [0] https://www.frontiersin.org/articles/10.3389/fcell.2023.1339390/full https://www.frontiersin.org/articles/10.3389/fcell.2023.1339....
  - hermitcrab 3y ago
    Would be interested to hear their side of it.
VoodooJuJu 3y ago
Crassness breeds clicks, and that's all they want.
- gs17 3y ago
  Who is "they"? The "researchers" wouldn't want to be known as frauds, they were probably hoping this would slip by quietly so they could list it as a publication and no one would ever read it. The reviewers are probably being harassed with inboxes full of AI rat phalluses right now. The journal wouldn't want to be made famous for publishing it. Frontiers has enough reputation issues without the most viewed paper of the year being this, and they don't get paid by readership, so they'll likely try to clean things up, missing out on the article processing fees from other Midjourney users.
gs17 3y ago
>How could this possibly pass peer review? Not just peer review, Frontiers claims "out editorial office conducts a pre-screening for validation of research integrity and quality standards" before that point. There's also supposed to be a "final validation" phase where they go over it again. It shouldn't have made it to peer review and shouldn't have made it out of it.

blibble 3y ago

> How could this possibly pass peer review? they've probably outsourced that to generative AI too

amyjess 3y ago
I wouldn't be surprised. I've seen people say that instead of reading articles now, they just open up ChatGPT and say "Summarize the important parts of the following article for me". My experience is that ChatGPT is laughably wrong when it comes to generating any kind of factual information. At best, it's good for feeding ridiculous prompts into it and trying to see what the most cursed thing you can get to spit out is so you can have a laugh.
- geoelectric 3y ago
  My experience is querying an LLM usually does ok if the only knowledge it needs is directly included in the prompt (including for summarization). It’s not perfect but its reliability is within the realm of human error and so isn’t significantly worse than me skimming the text myself. But it’s really unreliable if it’s leveraging “knowledge” from its training. Asking what something in an article means, for example, is pretty risky. Open-ended questions that cause knowledge dumps are very risky—the longer the answer, the riskier. Even text translation is risky, even though it’s ostensibly a simple transformation, because it has to draw the word mappings from training. And god help you if you ask it a leading question without realizing it. It’ll often just parrot your assumption back to you. To your point, though, using an LLM as an oracle/teacher would be a lot less dangerous if it were always laughably wrong but it’s usually not. It’s usually 90% right, and that’s what makes it so insidious. Picking out the 10% that isn’t is really hard, especially if you don’t know enough to second-guess it. The wrong stuff usually looks plausible and it’s surrounded by enough valid info that what little you do know seems to corroborate the rest.
gbjw 3y ago
The reviewers' names are listed on the publisher's (Frontier) site [0, top right] [0] https://www.frontiersin.org/articles/10.3389/fcell.2023.1339390/full https://www.frontiersin.org/articles/10.3389/fcell.2023.1339....
- hermitcrab 3y ago
  Would be interested to hear their side of it.

VoodooJuJu 3y ago

Crassness breeds clicks, and that's all they want.

gs17 3y ago
Who is "they"? The "researchers" wouldn't want to be known as frauds, they were probably hoping this would slip by quietly so they could list it as a publication and no one would ever read it. The reviewers are probably being harassed with inboxes full of AI rat phalluses right now. The journal wouldn't want to be made famous for publishing it. Frontiers has enough reputation issues without the most viewed paper of the year being this, and they don't get paid by readership, so they'll likely try to clean things up, missing out on the article processing fees from other Midjourney users.

gs17 3y ago

>How could this possibly pass peer review? Not just peer review, Frontiers claims "out editorial office conducts a pre-screening for validation of research integrity and quality standards" before that point. There's also supposed to be a "final validation" phase where they go over it again. It shouldn't have made it to peer review and shouldn't have made it out of it.