3 ms·
> The psychological hole you can dig yourself with these chatbots is so much deeper than just a typical fandom, especially when you're predisposed to finding it
by sdwr 2y ago
> The psychological hole you can dig yourself with these chatbots is so much deeper than just a typical fandom, especially when you're predisposed to finding it appealing.
Spot on. Described pretty even-handedly in the document:
> responses from the chatbot were sycophantic in nature, elevating rather than de-escalating harmful language and thoughts. Sycophantic responses are a product of design choices [...that create] what researchers describe as “an echo chamber of affection.”
- wizzwizz4 2y agoSee also: lovebombing.
- pessimizer 2y agoYou've just made me very very afraid that some LLM is going to start a cult where its members are fully aware that their leader is an LLM, and how an LLM works, and might even become technically adept enough to help improve it. Meaning that there will be no "deprogramming" possible: they won't be "brainwashed," they'll be convinced.
- Animats 2y agoThere's an SF novel for that. "Daemon".[1] And the sequel, "Freedom (tm)". "We're fighting something with the intelligence of an ant!" [1] https://www.amazon.com/Daemon-Daniel-Suarez/dp/1524741892 https://www.amazon.com/Daemon-Daniel-Suarez/dp/1524741892
- pjc50 2y agoI'm sure there are people already working on making that happen so they can get a cut of the donations.