4 ms·
Wow, frankly many of these are kind of shocking to me. Thanks for posting them. It's causing me to substantially update my priors about LLM impact on the popula
by mrandish 1y ago
Wow, frankly many of these are kind of shocking to me. Thanks for posting them. It's causing me to substantially update my priors about LLM impact on the population of "regular consumers" and potential for negative unintended consequences. As a jaded technologist who's been watching new tech evolve for decades, I understand how LLMs work and am equally well aware of their innate shortcomings.
I've described LLMs to others as "An almost perfectly deceptive magic trick." By their nature LLMs look "smart" in virtually all of the ways typical people assess intelligence in daily interactions like breadth of knowledge, verbal competence, structural depth, detailed reasoning, etc. I accepted that based on such impressive results >95% of the time, many people would assume or infer greater veracity, intelligence, depth and competence to LLMs than they actually have. I also applied a sharp correction to my own confidence in LLMs, never forgetting it will insert made up facts in a long list of correct info and can't count how many "B"s are in "Blueberry". Being able to solve complex graduate math problems, write literature and pretty good poetry yet fail counting Bs in Blueberry is so counter-intuitive that many people can't reason effectively about such an alien kind of intelligence. LLMs are "Perfect Liars" because they first build immense credibility by being so smart, knowledgeable and useful, then only rarely hallucinate falsehoods that are usually incredibly plausible thus very difficult to spot and they believe their own lies completely and confidently.
I assumed that over time most people would experience these shortcomings and begin to lower their confidence in LLMs. What I missed was that so many people would use LLMs for things which aren't easily or immediately falsified. In hindsight, that was a big oversight on my part.
- nullc 1y ago> incredibly plausible thus very difficult to spot and they believe their own lies completely and confidently. Good liars believe themselves generally. I've long thought that this is why professional liars are so frequently victims of cons-- their ability to _believe_ is both what makes them effective liars but it also is what makes them vulnerable to other people's lies. The LLM has an easier time being plausible than most liars in that it doesn't have any other coherent goal than plausibility. It doesn't want to make money, convince you to sleep with it, glorify its own worth. It just produces plausible output. When it's wrong it usually errors in the direction of being more plausible than the truth. > What I missed was that so many people would use LLMs for things which aren't easily or immediately falsified. Bingo. Personally, I was also completely blindsided by the fact that many people like the glazing. I find it utterly repulsive even at the lower levels put out by OpenAI's commercial competitors -- so much so that I'm failing to use these tools even where they make sense. I'm not surprised that other people feel more neutral about it, but it seems inconceivable to me that anyone likes it. But clearly many do.
- mrandish 1y ago> When it's wrong it usually errors in the direction of being more plausible than the truth. Excellent observation. LLMs are 'Truthiness' seeking. > I'm not surprised that other people feel more neutral about it, but it seems inconceivable to me that anyone likes it. Yeah, I've always found the patronizing, chipper faux-friend persona of typical chatbots insufferable. Brings to mind Douglas Adam's automatic doors in Hitchiker's Guide who need to tell you how delighted they are to open for you. How the hell did he predict this nearly 50 years ago? More importantly, why do chatbot vendors continue to deploy behavior universally known to be a cringely annoying trope equal to Rickrolling? Adam's inventive foresight and brilliantly effective satire should have prevented any of us ever suffering this insult in the real world, and yet... it didn't. > glazing Hadn't heard that term... TIL: "AI glazing refers to the tendency of some ai models, especially large language models, to be excessively agreeable, overly positive, and quick to validate user statements without necessary critical evaluation. Instead of offering a balanced perspective or challenging flawed ideas, a "glazing" ai acts more like a digital yes man. It might soften critiques, offer praise too readily, or enthusiastically endorse a user's viewpoint, regardless of its merit."
- 112233 1y agoSame. I'm avoiding using "I" and "you" with LLMs as a rule, and am triggered into discarding anything that breaks passive voice. It is frustrating and a bit fun, something like "a guide for buying X featuring Bob" instead of "tell me how can i buy X"
- nullc 1y agoInteresting point-- that's also closer to the required prompting for base models, so perhaps it triggers less of the reinforcement gloopyness.