6 ms·
Problem is people seriously believe that whatever GPT tells them must be true, because… I don't even know. Just because it sounds self-confident and authoritati
by Sharlin 9mo ago
Problem is people seriously believe that whatever GPT tells them must be true, because… I don't even know. Just because it sounds self-confident and authoritative? Because computers are supposed to not make mistakes? Because talking computers in science fiction do not make mistakes like that? The fact that LLMs ended up having this particular failure mode, out of all possible failure modes, is incredibly unfortunate and detrimental to the society.
- pousada 9mo agoI think in science fiction it’s one of the most common themes for the talking computer to be utterly horribly wrong, often resulting in complete annihilation of all life on earth. Unless I have been reading very different science fiction I think it’s definitely not that. I think it’s more the confidence and seeming plausibility of LLM answers
- Sharlin 9mo agoSure, but this failure mode is not that. "AI will malfunction and doom us all" is pretty far from "AI will malfunction by sometimes confabulating stuff".
- oneeyedpigeon 9mo agoPeople are literally taking Black Mirror storylines and trying to manifest them. I think they did a `s/dys/u/` and don't know how to undo it...
- direwolf20 9mo agoThey codysld start by trying to dysndo it. I'm sorry. That was a terrible joke.
- TeMPOraL 9mo agoThe stories I read had computers being utterly horribly right, which resulted in attempts (sometimes successful) at annihilate humanity.
- rsynnott 9mo agoIn terms of mass exposure, you're probably talking things like Cmdr Data from Star Trek, who was very much on the 'infallible' end of the fictional AI spectrum.
- TheSpiceIsLife 9mo agoData was also famous for getting things embarrassingly wrong, particularly when interacting with his human colleagues.
- pera 9mo agoLast year I had to deal with a contractor who sincerely believed that a very popular library had some issue because it was erroring when parsing a chatgpt generated json... I'm still shocked, this is seriously scary
- TeMPOraL 9mo ago"SELECT isn't broken" isn't a new advice, and it exists for a reason.
- Cthulhu_ 9mo agoI don't remember exactly who said it, but at one point I read a good take - people trust these chatbots because there's big companies and billions behind them, surely big companies test and verify their stuff thoroughly? But (as someone else described), GPTs and other current-day LLMs are probabilistic. But 99% of what they produce seems feasible enough.
- nullc 9mo ago> But 99% of what they produce seems feasible enough. This being a big part of the problem-- their false answers are more plausible and convincing then the truth. The output almost always seems feasible-- true or not is an entirely different matter. Historically when most things fail they produce nonsense. If they do they are producing something related to the truth (but perhaps biased or mis-calibrated). LLM output can be both highly plausible and unrelated to reality.
- anon_anon12 9mo agoPeople's trust on LLM imo stems from the lack of awareness of AI hallucinating. Hallucination benchmarks are often hidden or talked about hastily in marketing videos.
- cess11 9mo ago[flagged]
- IgorPartola 9mo agoTo quote Luke Skywalker: Amazing. Every word of what you just said is wrong.
- TeMPOraL 9mo agoWhich is why I keep saying that anthropomorphizing LLMs gives you good high-order intuitions about them, and should not be discouraged. Consider: GP would've been much more correct if they said "It's just a person on a chip." Still wrong, but much less, in qualitative fashion, than they are now.
- NoGravitas 9mo agoJust a weird little guy.
- dematz 9mo agoA hungry ghost trapped in a jar (https://bsky.app/profile/hikikomorphism.bsky.social/post/3lwamjk3pjk2x https://bsky.app/profile/hikikomorphism.bsky.social/post/3lw...)
- TeMPOraL 9mo agoSimilar analogy, yes. FWIW, I prefer my "little people on a chip" because this is a deliberate riff on SoC, aka. System on a Chip, aka. an actual component you put when designing computer systems. The implication being, when you design information processing systems, the box with "LLM" on it should go where you'd consider putting a box with "Person" on it, not where you'd put "Database" or any other software/hardware box.
- Suzuran 9mo agoMy boss says it's because they are backed by trillion dollar companies and the companies would face dire legal threats if they did not ensure the correctness of AI output.
- buggy6257 9mo agoYour boss sounds hilarious naive to how the world works.
- Suzuran 9mo agoIn a lot of ways he is, despite witnessing a lot of how the sausage is made directly. Honestly, I think at at least half of it is wanting to convince himself that the world still functions in ways that make sense to him rather than admit that it's mostly grifters grifting all the way down.
- nathanaldensr 9mo agoThe high-trust Boomer brain cannot comprehend the actual low-trust society of grifters in which we live.
- deleted 9mo ago[deleted]
- pluralmonad 9mo agoI don't agree with this blanket statement. The internet is low trust for lots of reasons, but regular (read small, proximal/spatiotemporally constrained) communities still exist and are not grifters all the way down. Acknowledging that distant strangers are not trustworthy in the traditional sense seems reasonable, but is categorically different than addressing natural social groups (small and local).
- OGEnthusiast 9mo agoYes, and most young Americans are locked out of those small, high-trust suburbs due to high housing prices. So instead they get to experience the magic of low-trust America first-hand, hence the disconnect between the young and the boomers.
- tveita 9mo agoI think people's attitude would be better calibrated to reality if LLM providers were legally required to call their service "a random drunk guy on the subway" E.g. "A random drunk guy on the subway suggested that this wouldn't be a problem if we were running the latest SOL server version" "Huh, I guess that's worth testing"
- TheSpiceIsLife 9mo agoThere’s a non-zero number of people who would get a chuckle out of a browser extension at replaces every occurrence of LLM or AI with a random drunk guy on the subway .
- direwolf20 9mo agoIt could be the same extension that replaces every occurrence of the cloud with my butt.
- TheSpiceIsLife 9mo agoThat’s the one I trying to think of, I could remember the ‘butt’ part.
- pjc50 9mo agoBillions of dollars of marketing have been spent to enable them to believe that, in order to justify the trillions of investment. Why would you invest a trillion dollars in a machine that occasionally randomly gave wrong answers?
- johnnyanmac 9mo agoThis is probably more of a GAI achievement, but we definitely need confidence levels when it comes to making queries with factual responses. But yes, look at the US c.2025-6. As long as the leader sounds assertive, some people will eat the blatant lies that can be disproven even by the same AI tools they laud.