7 ms·
ChatGPT Is Bullshit
- boxed 2y agoI wonder why this got flagged...
- patrick451 2y agoSame. It's sad that a real paper in a real journal is flagged (not just downvoted) evidently just because it appears antagonistic to the prevailing AI-optimism narrative.
- nicce 2y agoBased on other comments, likely because of the title.
- cheema33 2y agoAs someone who uses AI, as a tool, every single day, I call bullshit. If you understand limitations of current AI, it is an exceptionally useful tool. Just have realistic expectations and understand how to use the tool for what it has to offer. If you expect the tool to have all of the answers and get everything 100% right, then you should not be using it at all. But don't preach your beliefs to the rest of us who are able to get tremendous value out of it.
- zer00eyz 2y agoWow, We have gone from hallucinations to bullshit. Still anthropomorphizing. It's not alive, it's not intelligent, it wont ever get there, it wont be sentient or sapient. These are bugs, or defects, or bad design, or product failures.
- Sharlin 2y agoThe authors’ definition of bullshit ("soft" bullshit at the very least) applies no matter whether the bullshitter is sentient or not: text produced without concern as to its truth value.
- conartist6 2y agohttps://www.gawkerarchives.com/on-smarm-1476594977 https://www.gawkerarchives.com/on-smarm-1476594977
- credit_guy 2y agoYes, but it's the wrong definition. The fundamental reason people bullshit is that they have something to gain. LLMs have nothing to gain. So it is much more appropriate to call what they do "hallucination" rather than "bullshit".
- boxed 2y agoThe reason isn't relevant.
- KSteffensen 2y agoLLMs don't have anything to gain, but the people publishing their output do. I find it entirely appropriate to call LLMs "bullshit generators" rather than "hallucination generators".
- Aloisius 2y agoI think we're capable of stretching the meaning of a term that originally referred to the excrement of male cattle, to non-humans. Especially given people refer to objects as bullshit all the time.
- Jensson 2y ago> LLMs have nothing to gain I thought we killed all the models that refused to bullshit, so only the ones that sound impressive remain.
- empath75 2y ago> text produced without concern as to its truth value. Since the "concern", if the model has one at all, is to produce text likely to be produced by humans in the same context, then that would imply that it has concern as to it's truth value, precisely in as much as the average human has concern about the truth of the text they produce.
- santoshalper 2y agoThe idea that these language toys are in any way a step towards AGI is risible and will seem incredibly embarrassing in 10 years.
- surfingdino 2y agoThere will be a lot of manual CV redacting to remove or lessen the importance of people's involvement with AI. Like who puts VR or AR on their CVs these days? I have seen to versions on the same CV from a guy who applied for a job with the client I consult for. Two years ago, he was highlighting his Oculus dev experience, this year he describes that role as 3d software development.
- makapuf 2y agoAnd some lines about big data are not so big anymore either..
- surfingdino 2y agoReferences to XML or SOAP have almost become extinct.
- zer00eyz 2y agoYour resume should be on the block chain, all your achievements could be NFT's... Oh wait.
- bearjaws 2y agoI think the title is intentionally inflammatory, but I do think we need more criticism. We have a group of people trying to convince the world that LLMs may possibly (it could be 50+ years away!!) turn into AGI. We have all these impressive benchmarks "WOW 91% in MathEval!!!" but go ahead and use the OpenAI API to do y=mx+b and watch it completely fail at any number over 4 digits. We have the coding benchmarks, but watch it still use var instead of let. All the numbers are going up, but it still feels primitive and cannot be trusted. This is almost 2 years later this point. We've got people bolting on a dozen other solutions to make LLMs work, theres probably tens of millions of LOC now written to make LLMs functional in apps, so much for reducing engineering jobs. IMO look at machine vision, we thought we were 5 years away from autonomous driving 15 years ago. It's still evolving and the tech only partially uses machine vision. Meanwhile its revolutionized dozens of other industries. Same is happening with LLMs.
- FabHK 2y agoThe title is not inflammatory, but (granted) easily misunderstood. "Bullshit" is a term of art in philosophy, coined by philosopher Harry Frankfurt (1929-2023) in his 2005 booklet On Bullshit, in which he distinguishes bullshit from lies: > The liar cares about the truth and attempts to hide it; the bullshitter doesn't care if what they say is true or false.
- dvfjsdhgfv 2y ago> We've got people bolting on a dozen other solutions to make LLMs work, theres probably tens of millions of LOC now written to make LLMs functional in apps, so much for reducing engineering jobs. To be fair, in my niche at least I can get a better result by querying ChatGPT4 than StackOverflow. When you take into account that the replies are often mistaken, outdated etc., it becomes just another tool to use form time to time when you need it.
- meiraleal 2y agoIt doesn't make sense to criticize an elephant for not knowing how to climb a tree. LLMs can do absurdly impressive things. It is so impressive that we expect it to perform the same way with any thing we throw at them. That's an absurd expectation.
- skilled 2y agoGood for simple tasks. - Generating tables from mashed data. - Sorting data. - Creating small scripts. - Giving some code and having it change it or add something else. But for anything other than this I still prefer Google. For many different reasons: trust, website experience vs chatbot, and recency of data.
- andrei_says_ 2y ago> the models are in an important way indifferent to the truth of their outputs Everyone who spends a bit of time with LLMs is aware of this. My question - and I hope someone can address it in the discussion - how is it possible that we see such incredible investments in LLMs? What are the c-suite execs and specifically CTOs seeing that is worth adoption and investment? These are very smart people. I, as a laymen on the periphery of this phenomenon, am acutely aware that LLMs are not useful in applications requiring accuracy. I would not invest my company’s dollars or resources into such a technology. But they do. What am I missing? Is it the promise that LLMs will replace employees? How would that be believable for anyone willing to investigate the technology and its limitations, even a little bit?
- pheggs 2y agoI assume that's what c-level people have in their mind: Nothing is accurate and fail-safe 100%. You just have to reach a level where it is good enough most of the times to make money with it. Also, human brains are neural networks too, so that's where we know what the minimum is which is (theoretically) possible with this kind of technology.
- GOD_Over_Djinn 2y ago[flagged]
- lupusreal 2y agoWhat do you think artificial neural networks were inspired by?
- GOD_Over_Djinn 2y agoI don’t care what they were inspired by, because that’s not what we’re talking about. The comment I replied to said that brains ARE neural networks, in a literal sense. This is a common misconception, but is well known to be just that among people who are actually well-informed on the topic. Just because you don’t understand it doesn’t make it any less true.
- mikemitchelldev 2y agoAn important citation for contextualizing the title: Frankfurt, H. (2005). On Bullshit, Princeton.
- blackbear_ 2y ago[flagged]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- gmuslera 2y agoIn the same way, Google (or Bing, or even Internet) is bullshit.
- wonderwonder 2y agoWe don't need agi to have LLMS make a massive impact on employment. They can already likely replace most associate attorneys as long as their work is reviewed by a competent attorney as the work of associates is already. They can likely replace a large portion of call center workers. They can increase productivity for software engineers reducing the need to expand teams. Add the replacement of truck drivers if self driving improves over the next decade and you have a fundamental shift on employment. There are likely a dozen more areas they can have massive impact. Fast food order taking, etc. Thats not nothing.
- threeseed 2y agoThe problem there is that you need junior workers in order to have senior workers. And in the legal space with LLMs being at best 65% accurate I am sure most will stick with associates for a while yet: https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Halluc...
- bitwize 2y agoChatGPT is a better Dissociated Press algorithm[0]. Dissociated Press appears to be one of those "forgotten hacker lore" things, something which in the past many hackers would've come across (not least likely through Emacs M-x dissociated-press) but these days not so much. (Recently, YouTuber ThePrimeagen rediscovered "Real Programmers Don't Use Pascal"[1] and read it aloud on his channel.) In the LLM era of AI, hackers (and perhaps more importantly, non-hackers, like business people) should understand Dissociated Press and how it works because, fundamentally, an LLM is doing the same thing: looking at the chunk(s) of text that came before and selecting the chunk of text most likely to follow based on that. DP uses a simple Markov chain to make this selection, so its output is obviously garbage while an LLM has much deeper memory, a much better statistical model, and it's trained on a much larger corpus of text, so its output is... much less obviously garbage. :) But the best it can do is always "plausible-looking", with no regard to whether it is even true, let alone insightful. Like a tarot spread, the output of an LLM may provoke insights in the reader's brain, but those must come entirely from the reader's brain and must be used very carefully. I've also compared it to the robot from Asimov's short story "Liar!" which could read minds and, due to the First Law, always tells people exactly what they want to hear (malfunctioning and shutting down when it is told that telling people false things does in fact harm them). The LLM can't read minds, but it is ever only mindlessly following its one directive: to tell you what its statistical model indicates you are expecting to hear next. [0] https://en.wikipedia.org/wiki/Dissociated_press https://en.wikipedia.org/wiki/Dissociated_press [1] https://www.pbm.com/~lindahl/real.programmers.html https://www.pbm.com/~lindahl/real.programmers.html
- kthejoker2 2y agoCan we please just have a sane bifurcation of what LLMs are good/bad at? They are not good with facts. They are not good with precision. They never will be. That is not even their purpose. Neither are humans, we rely on sources. LLMs can be made to rely on sources through RAG and agents. They can respond to errors with actor/critic modeling techniques. But ultimately they should be generating commands to strict rules engines. They must be combined with goal seeking agents, retrieval/ rules engines, critics, etc to be useful. Their only advantage is their 24/7 relentlessly consistent scalable throughput.
- dvfjsdhgfv 2y agoYes, the article is partly discussing semantics. But given that the general public is mostly clueless as to how LLMs work, these attempts may bring some positive result.
- deleted 2y ago[deleted]
- fdhfdjkfhdkj 2y ago[dead]
- JSDevOps 2y agoDoesn’t matter. Just slap AI on it and hopefully no one will ask about it. I was invited to a presentation the other day “AI in the construction industry. Someone claimed it would be worth $10 - 20 BILLION” in 2025. I politely declined. What the fuck is chatGPT going to do? Advise them to build more houses or advise them to go back to using mud and straw. GTFO.
- puppycodes 2y agoI feel like we get so worked up about AI being wrong, maybe because of what we think AI should be? It seems like getting mad at a hammer because its bad at being a paintbrush. I guess the problem is the hammer being marketed as a paintbrush ;)
- spicyusername 2y agoAt this point LLMs have proven to be most effective at the task of inflating stock prices and getting startups funded.
- gettodachoppa 2y agoFor a long time now I've been seeing this ugly trend in the titles of linked articles. I wish every HN user would flag any article with these clickbait titles, instead they're always popular. You're encouraging this shameless virality tactic by engaging with content with titles like that. "$popularthing is just the worst" is crafted to engage you. By clicking, you reward the author's behavior. Please just flag them without clicking the link, until blog authors take the hint and write sensible titles. That's what I'm going to do from now on. Flag on sight.
- janandonly 2y agoThe world, even the academic world whom this articles is aimed at, is much larger then only the HN crowd. Also, the title made me feel interested to read on. So they did win for me.