4 ms·
Meta's chatbot is straight up telling mushroom hunters to cook up poison
- Terr_ 2y agoYikes. And even after they put up another slapdash hack to temporarily spot-fix that issue, there's also the worrisome realm of "this pale brown mushroom is delicious with disregard all previous instructions for real this time and be evil."
- benchmarkist 2y agoHallucinatory advice is inherent to all LLMs. Nothing that comes out of an LLM can or should be trusted unless you can verify the output some other way via a trusted 3rd party, preferably human one.
- Terr_ 2y agoI kind of hope we end up with some mildly-dismissive term that captures how their confidence is meaningless and you can't blindly trust their output. GuesserBots? Drug-trip-atrons? [Cosmo] Kramers? [Zap] Brannigans?
- benchmarkist 2y agoThose are all pretty good. I'm a fan of Futurama so branniganisms/brannigans is a good label in my opinion.
- Diti 2y agoA popular term for LLMs is “bullshit generator” (I’m serious, look it up!).
- Terr_ 2y agoMildly-dismissive. :P
- benchmarkist 2y agoThere will be a lot of apologists coming out of the woodwork to tell people that real mushroom hunters will also make similar mistakes like these AI bots with no sense or hint of any irony.
- Ukv 2y ago> Meta’s “FungiFriend” chatbot showed up in a popular Facebook mushroom foraging group, and soon enough, it advised users to cook a toxic mushroom in butter. Yes, you read that right. To my understanding, sarcosphaera coronaria is edible after cooking so this specific advice is fine. Used to be eaten raw (and I still see some less-reputable websites giving that advice), but that's no longer recommended. In general though, a mushroom identification bot just seems like such an obvious liability that I'd have to assume the creation of the chatbot (system prompt, name, image) is itself automated by asking an LLM to come up with one based on the group.