9 ms·
Family poisoned after using AI-generated mushroom identification book
- rationalfaith 2y ago[dead]
- juanani 2y ago[dead]
- tptacek 2y agoDo we believe this story? It's a Reddit "LegalAdvice" thread. It's not morel season in the UK right now; false morels are also March-May produce.
- fsckboy 2y agoI searched for the title of the book and nothing came up. When does searching for something NOT bring up amazon links?
- maxbond 2y ago> The book is entitled something similar to: "Mushrooms UK: A Guide to Harvesting Safe and Edible Mushrooms." > The book has been removed from sale from the online retailer, ... Emphasis added. I don't know if this is real or not, but they haven't given us the title.
- bee_rider 2y agoIt sounds like it wasn’t an Amazon product. Maybe it is some very niche UK site or something.
- JCW2001 2y agoAmazon sells all kinds of crap. I doubt there's any reason they wouldn't stock it.
- pflenker 2y agoThe poster doesn’t claim they ate morels. The section about morels was an example for something unrelated to the poisoning.
- tptacek 2y agoThe fact that these people claim mushroom poisoning but don't say what kind of mushrooms they were looking for makes you... more? less? willing to buy the story? They mentioned morels, which are sort of plausible (dates excepted) since false morels aren't going to kill you. What kind of mushroom would they be hunting in late July where their poisoning wouldn't be actually-newsworthy?
- pflenker 2y agoI really have no opinion here at all. I just wanted to point out that the fact that it’s no morel season in the UK is no evidence in either direction since the post doesn’t claim that the poisoning was due to morels. I saw a comparable post the other day about an LLM proposing to mix garlic and oil in a way that would have produced a poisonous fluid, which the author recognized before trying. I found that story, and I find this one, at least plausible.
- bee_rider 2y agoThey might be novices who got the book as the start of their search, right? Since it could have any type of AI hallucinated mushroom with any type of AI hallucinated description. Are there really no non-deadly mushrooms in the UK in the late July? They don’t necessarily have to be plausible or near misses or anything, right?
- grugagag 2y agoHey, this is a book published with errors generated with LLMs. You can look it up
- wcedmisten 2y agoThere have been quite a few articles corroborating the existence of books like this, the only part that needs believing is that someone actually used the books as advertised. https://www.vox.com/24141648/ai-ebook-grift-mushroom-foraging-mycological-society https://www.vox.com/24141648/ai-ebook-grift-mushroom-foragin... https://fortune.com/2023/09/03/ai-written-mushroom-hunting-guides-sold-on-amazon-potentially-deadly/ https://fortune.com/2023/09/03/ai-written-mushroom-hunting-g...
- tptacek 2y agoRight, that's the only thing I'm questioning.
- rurp 2y agoEven if you doubt this particular case (which I haven't seen any reason to do), it's inevitable that people will believe some of the things they read in legitimate looking books sold by major online retailers.
- ggjkvcxddd 2y agoThe fact that this exact thing has received media coverage increases the likelihood that the post is fake. Of course the story remains plausible, and it certainly has an air of truthiness, but I'd give it very good odds for being fake.
- samtheDamned 2y agoI would take any story from reddit with a huge grain of salt, but even if this case is not real there are real AI generated books that pretend to be real books, and mycology books in particular have had attention brought to them due to the dangers inherent to the misinformation they may contain.
- rachofsunshine 2y agoYeah, while this story is absolutely plausible, plausible does not necessarily mean true. OP is a new account with suspiciously few details that would permit fact-checking, and I think I'd call this one ragebait until proven otherwise. (That does not mean that this couldn't happen or that there isn't a risk of it happening, but everyone here hopefully knew that before this thread. I'm saying it shouldn't adjust your beliefs much if at all.)
- alwa 2y agoI share your suspicion of the thin details. And what details they do provide smell funny too. I’m especially skeptical of this supposed cloak and dagger business: > My wife just received an email from the online retailer. She has been asked to "Not take any photographs or copies of the product in question due to copyright issues" and it states, "the product must be returned immediately by special delivery by [DATE]." There's some other statements as well about our account being terminated if we fail to return the product by the specific date. We've got a lot of movies and series that we have purchased over the years on this account, I wouldn't want to lose them.
- mjhay 2y agoYeah, somebody wanted some internet points.
- paulgb 2y agoI generally distrust Reddit threads, but it's entirely plausible to me. I was once gifted a cookbook on Amazon that was full of pre-LLM "sludge" of suspect internet-collected recipes along with a stock photo of an author with fake credentials (her author page is still up, although her books have been pulled and apparently she's a man now https://www.amazon.com/stores/author/B0716Q2Y4Z/about https://www.amazon.com/stores/author/B0716Q2Y4Z/about) When I left a negative review pointing out that the author was a stock photo (entire content: "The author of this book is a fraud. There is no Tina B. Baker, she is a stock photo."), Amazon pulled the review saying it violated their guidelines.
- alexey-salmin 2y agoSure, but a fact of a story being plausible not only increases a probability of it being true, but also increases a probability of someone faking it because it's so plausible. Chances that someone would do exactly that on reddit are very high.
- madaxe_again 2y agoThe bit where they say Amazon demanded that they return the book by special delivery or face having their entire Amazon account terminated is where the point at which it became clear to me that this is ragebait - Amazon simply do not work that way in the U.K. - returns are via prepaid label that they provide, and they don’t terminate your account if you fail to return a product, you just don’t get refunded.
- meroes 2y agoIt’s probably AI generated like most of the rage bait on Reddit.
- grugagag 2y agoThat’s to be expected from genAI. Im sure more of this is to follow and these AI companies will have to come up with disclaimers and checkoff boxes (CYA type) “I understood these answers are likely wrong.” Even better, they should mention LLMs do not understand what they generate. Edit: Wait, this is a book made up with LLMs. I think the author should be on the hook for publishing unless they added a disclaimer their book has no grounds in reality.
- samtheDamned 2y agoThis was something that was bound to happen and is going to happen again unless we get more serious regulation around AI publishing. > It lists the author as having a Masters Degree in Mycology from University of East Ontario. A search later revealed there is no "University of East Ontario." This has got to be criminal negligence.
- rdtsc 2y agoWhat's good for the goose is good for the gander: the AI generated not only the contents but the author as well. This has to be some kind of a new level of idiocy. I mean, use AI to make junk sci-fi stories, and generate fake authors all day. But going for a book about mushrooms deserves a special stupidity and evilness award.
- deleted 2y ago[deleted]
- throwanem 2y agoIt probably could be automated to a point where there's no human even in the loop here: I wonder if books about mushroom identification are trending in Amazon UK sales lately, or something like that. No excuse, obviously. But it could explain how whoever's running the scam could have run it off this cliff - probably others, too. It's been a couple years since I bought even paper books off Amazon. Used should theoretically be closer to OK, but eBay sellers are cheaper and much more trustworthy.
- kalaksi 2y agoSounds more like a scam.
- isodev 2y agoIsn’t this the case for almost everything “AI”? Regulation like the EU’s AI Act (and even more strict) should be in effect worldwide. Corps have been running the show but naturally their focus is on monetisation rather than what their creations actually do.
- CM30 2y agoThis is why generative AI simply doesn't work for anything with actual consequences. As it is right now, it's a glorified autocomplete, and one that doesn't understand the context it's being used in at all. That's fine for things like stock images or text on random product pages, but for things like this? Yeah, the very concept is just risky as hell.
- SkyPuncher 2y agoSearch also doesn’t work for anything with actual consequences by this logic. You verify information by finding multiple, different sources.
- dylan604 2y agobut at this point, a committed actor would be able to generate multiple, different sources for this very reason
- puppycodes 2y agoThis is actually a problem with non-AI generated mushroom books as well. Apparently publishers have been copying eachothers misinformation for a long time.
- politelemon 2y agoTrap mushrooms
- rnhmjoj 2y agoIt is also a problem with accurate images and descriptions. Some mushrooms are very easily mistaken for others: unless you're 100% sure you've picked this very species before in around the same place and time, you'd better leave it there or bring it to a mycologist for identification.
- im3w1l 2y agoHow long until dna tests become cheap and available enough to use for mushroom identification? I wonder what that will do to the hobby...
- pvaldes 2y agoThe dna found belongs to a mushroom called "common garden snail" plus "unknown". The database says that common garden snail is a delicacy so the stuff is fine to eat. Good appetit!.
- mcbuilder 2y agoI agree, I always thought eating mushrooms required an expert for confirmation, otherwise you are rolling the dice. Buying a random book off Amazon and using that seems like the very definition of something a non-expert would do.
- culebron21 2y agoYou simply steer away from those that are easy to misidentify. There aren't that many species in any given area, and those that look like poisonous are few, and well known. In my childhood, when we lived in real suburbia, with woods in 5 minutes of walk or right at the door, we routinely gathered mushrooms, and never had a problem. We stopped it when we moved to other places, or probably because adolescence came around -- probably, parents had taken us for an interesting kind of walk, and then we'd not want it anymore. Right now I can only recall some names of mushrooms and only sure of one species, so I won't risk at all.
- stonethrowaway 2y agoI’ve become suspicious of any book being published after 2022 as a result of AI-generated content. Wherever possible I opt to find an older edition or a book that covers a subject that’s pre-AI, and hopefully containing an errata to boot.
- aaroninsf 2y agoThis has got to be settlement-chasing, directly akin to the people who throw themselves into stopped cars in Russia for insurance fraud purposes, only to be captured on dash-cam video and derisively immortalized on social media. The other viable explanation is "but Google Maps told me to drive off the pier."
- bee_rider 2y agoIt seems too random to be a scam, the perpetrator needs to find a mushroom book with an error in it to do the scam in the first place.
- Tagbert 2y agoI see this happen a lot on Tiktok and Facebook where influencers post about using different plants for food or for skincare. They often either show photos that are of a different plant that is dangerous to eat or they haven’t actually tried this and promoting the consumption of a plant that can poison people. It’s the Wild West out there on social networks.
- ChrisMarshallNY 2y agoObligatory PA Comic: https://www.penny-arcade.com/comic/2024/05/27/a-very-particular-set-of-skills https://www.penny-arcade.com/comic/2024/05/27/a-very-particu...
- Freak_NL 2y agoOuch. That web comic did not age gracefully.
- throwanem 2y ago'Age?' It's three months old.
- alwa 2y agoI read the parent comment to contrast the 3-month-old vintage of Penny Arcade with its first couple of decades: https://www.penny-arcade.com/archive/ https://www.penny-arcade.com/archive/
- throwanem 2y agoOh. Sure, I guess that's a take. Expecting the artist and writer to have learned nothing of their craft in the 26 years they've spent pursuing it as a profession seems extremely silly to me, but if someone finds that a sensible way to look at the work, I don't see what it would benefit anyone for me to try to gainsay them.
- 2y ago
- paxys 2y ago99% of all such "advice" subs on reddit are creative writing. I'm gonna hold off on the outrage till there's actual proof that this happened.
- inputError 2y ago[dead]
- cowboylowrez 2y agoIf its creative writing its for a worthy cause since the books do exist, theres plenty of warnings all over the web about these books written by AI and thats pretty outrageous right there. Its wierd that something like "artificial intelligence" actually makes everything stupider haha
- perihelions 2y agoHere's a more credible article about the general phenomenon (more so than an anonymous anecdote in "legal advice" Reddit, which is, by reputation, a place for amateur writers to hone their craft in a low-stakes environment). https://www.washingtonpost.com/technology/2024/03/18/ai-mushroom-id-accuracy/ https://www.washingtonpost.com/technology/2024/03/18/ai-mush... ("Using AI to spot edible mushrooms could kill you") - "Like past mushroom identification apps, the accuracy is poor, Claypool found in a new report for Public Citizen, a nonprofit consumer advocacy organization. But AI companies and app stores are offering these apps anyway, often without clear disclosures about how often the tools are wrong." - "Apple, Google, OpenAI and Microsoft didn’t respond to requests for comment."
- jowdones 2y agoWell the morale is: don't eat wild mushrooms no matter the source (pick them yourself or picked by an "expert"). At least that's my rule. There's a ton of cultivated mushrooms and a ton of other tasty safe foods to choose from, I don't give a fuck how exotic the wild mushrooms flavor would be. I'm not curios. Because curiosity killed the cat.
- jMyles 2y agoIt's wild to me that these sorts of stories provoke - on HN of all places - frothy calls for state intervention. Is it by design? Is this some dark campaign to stifle open source LLM development? Some mushrooms are horribly poisonous. That's the nature into which we were born. It's not a consequence of policy, and won't be fixed by policy. People need to learn to be careful about sources of information, with care in proportion to consequences. State intervention will preclude that evolutionary step, almost certainly without actually ameliorating the problem.
- deleted 2y ago[deleted]
- meroes 2y agoI’ve watched so many communities devolve because of rage bait, which is often AI generated. I doubt any community is immune. It seems to be an inherent weakness that properly targeted ragebait can subsume them.
- 34679 2y ago[flagged]
- tqi 2y agoIt's going to be hilarious when this poster posts a follow up complaining about AI generated legal advice they received on Reddit.
- 999900000999 2y agoWhy are you eating wild mushrooms ? Even if no one did anything wrong, you might misidentify it in the field. Unless you're very experienced in this field this feels like a very risky and stupid thing to do. This isn't an AI problem. This is a "Don't eat things growing in the woods" problem.
- kibwen 2y ago> Why are you eating wild mushrooms ? Perhaps they asked ChatGPT and were told it was a great idea to eat wild mushrooms. > This isn't an AI problem. This is a "Don't eat things growing in the woods" problem. This is a misinformation problem, which you can't solve simply by saying "you should have been better informed". The whole problem of AIs accelerating the post-truth age is that reliable sources are becoming scarcer at an exponential rate.
- RIMR 2y agoAmazing that we're at the point where people think "Don't eat things growing in the woods", something that would have doomed our species, is common sense.
- puppycodes 2y ago"This one secret doctors hate..." because thats where food comes from, not the grocery store stork
- thisisauserid 2y agoTo be fair, even the National Audubon Society Field Guide to North American Mushrooms tells you that if you follow it without knowing what you're doing your going to get poisoned. Not necessarily an AI issue.
- sitkack 2y agoThis is a shitpost, but everyone needs to be vigilant when they put anything inside their body.
- deleted 2y ago[deleted]
- steelframe 2y agoI generally treat anything on Reddit as first and foremost motivated by a desire for fake Internet points rather than by a desire to share a real story or have a real conversation. I'd at least expect a reputable journalist to pick up on this story if there really was anything to it. Regardless of the veracity of the claims in that post, there's not much new here aside from the fact that the distributor generated the content using AI rather than making it up themselves. Quackery and snake oil has always been a thing, and plenty of people have been seriously injured or died from misinformation about food safety or medicine. The next time someone hesitates to seek professional medical attention for a problem because they got a blessing from the elders at their church and they think God will heal them as soon as they start having more faith, we can start talking about where we can really draw the line between personal responsibility and holding liars liable.
- Sverigevader 2y agoDylan Beattie talked about this exact thing at NDC this year. https://youtu.be/By4Gb1RKZpU?t=1428 https://youtu.be/By4Gb1RKZpU?t=1428. Timestamp included. The entire talk is very good though.
- dvh 2y agoI am of the opinion that you should only be hunting mushrooms if your family has been hunting mushrooms for generations (I could count 4+ in my family). Or at least spend few years under some patronage. This is not something you should learn from books. Also you can't just learn 1 mushroom, you need to know them all even if you only eat one.
- haunter 2y ago>My wife just received an email from the online retailer. She has been asked to "Not take any photographs or copies of the product in question due to copyright issues" and it states, "the product must be returned immediately by special delivery by [DATE]." >There's some other statements as well about our account being terminated if we fail to return the product by the specific date. We've got a lot of movies and series that we have purchased over the years on this account, I wouldn't want to lose them. This story is so fake it hurts. Reddit eating up ragebait is one thing but posts like this doesn't belong to HN at all
- floodle 2y agoDoes it really matter if it's fake? It's a plausible scenario, and worth discussing. Fiction books can raise interesting questions about science and technology. And the implications of AI is very much HN's wheelhouse.
- deleted 2y ago[deleted]
- amatecha 2y agoDefinitely a "pics or it didn't happen" kind of post, IMO.
- aaron695 2y ago[dead]
- kamaal 2y agoPeople know "AI" is just software generating words it deduces should probably occur one after other right? Its not a fact searching exercise.
- 2-3-7-43-1807 2y ago[flagged]