6 ms·

Do you feel that you may be being a bit provocative by calling LLM's 'bullshit machines'? I understand the frustration as I've been bullshitted by these models

Do you feel that you may be being a bit provocative by calling LLM's 'bullshit machines'? I understand the frustration as I've been bullshitted by these models just as much as the next programmer, but surely with recent advancements in RAG and reasoning, they're not just 'bullshit machines' at this point, are they?

anon373839 2y ago
“Bullshit” actually means something: > In philosophy and psychology of cognition, the term "bullshit" is sometimes used to specifically refer to statements produced without particular concern for truth, clarity, or meaning, distinguishing "bullshit" from a deliberate, manipulative lie intended to subvert the truth. https://en.m.wikipedia.org/wiki/Bullshit https://en.m.wikipedia.org/wiki/Bullshit It’s really an ideal term to describe what LLMs do.
- HPsquared 2y ago
  I prefer "waffle" https://en.m.wikipedia.org/wiki/Waffle_(speech) https://en.m.wikipedia.org/wiki/Waffle_(speech) "Waffle machines" is even kind of funny.
  - deleted 2y ago
    [deleted]
  - fzzzy 2y ago
    Waffle machines is way better. Love it. Thanks.
  - m0llusk 2y ago
    Makes me imagine coming up to the hindquarters of a bull with a waffle machine on an extension cord.
  - burch45 2y ago
    Waffle means something very different though in the U.S., to “flip-flop” on a position. Not hold it for any fundamental reasons. But I don’t think you can say that an LLM holds a position whatsoever. Also, https://en.m.wikipedia.org/wiki/On_Bullshit https://en.m.wikipedia.org/wiki/On_Bullshit the essay originally popularizing the definition of bullshit considered here. Note the references to LLM output at he bottom.
- TeMPOraL 2y ago
  > It’s really an ideal term to describe what LLMs do. Only when you relax it to the point it also describes what most people do. To be specific: if by "truth" you mean objective, verifiable truth, and by "caring about truth" you mean caring about objective, empirically verifiable evidence, then by far most people are mostly only ever trying to be persuasive.
  - anon373839 2y ago
    I hope you don't actually mean that. It's a cynical, sad, and - quite frankly - false thing to believe.
    - TeMPOraL 2y ago
      Why? I don't have a problem recognizing that "being convincing" to the right people is a very good way of learning and verifying knowledge about objective reality. Just because it's by proxy, doesn't mean it doesn't work. This is how every one of us learns most things, and what's really sad to me is deluding yourself into thinking we're more special than that.
      - anon373839 2y ago
        Honestly, it just sounds to me like you’ve come up with some axioms in your mind about the “fundamental nature” of humanity - and then granted yourself the luxury of certainty about them. That, with a dash of misanthropy, is something I’ve been seeing more and more on HN these days.
BrenBarn 2y ago
Yes, they are.
immibis 2y ago
It is literally a machine that says what you want to hear. Was anyone in this thread around for Talk to Transformer? Remember the unicorns demo? They may have now figured out how to wrangle the technology so it's more likely to spit facts, but it's still the same thing underneath.
aqueueaqueue 2y ago
I have been thinking about this and I have an idea for local LLM I want to try. It is based on the assumption LLMs make mistakes (they will!) but be confident about it. Use a 1B model and ask it about a city for fun mistakes in an otherwise impressive array of facts. Then you will see how bullshitty LLMs are. Anyway the idea is to constrain the LLM and a language understanding and to choose a constrained response. For example give it the capability to read out something from my calendar. Using the logits it gets to choose what that calendar item is. But then regular old code does the lookup and writes a canned sentence saying "At 6pm today you have a meeting titled $title". This way, my meeting schedule won't make my LLM talk like a pirate. This massively culls what the LLM can do but what it does do, is like a search and so just like Google (before gen AI!) you get a result but you can judge it as a human.
- IceDane 2y ago
  This is basically how a lot of it works already. What you are describing is more or less just structured output and tool usage.
  - aqueueaqueue 2y ago
    Yes! Thanks for the terms. I guess I am saying restrict it to that. Probably how Siri etc. works. This gives a low bullshit usage pattern.
mvellandi 2y ago
Or are they modern day oracles? The full title is inclusive and invites consideration. Besides, how many different types of models are there? How are they used? For freely available ones made accessible to the general public via interfaces, when were they last updated? Do the interface implementors even care or was it a simple project to make money via ads and microtransactions? This is a new area of media literacy and requires critical thinking. Nonetheless, I do acknowledge the value of RAG-based approaches that attempt to qualify their reasoning through provided sources.
- CharlesW 2y ago
  > Or are they modern day oracles? Neither, which is probably why the parent commenter considers it provocative. "A bullshitter either doesn't know the truth, or doesn't care. They are just trying to be persuasive." In any case, this kind of anthropomorphization is definitely bullshit.
CyMonk 2y ago
they define the term bullshit in lesson 2: [quote]BULLSHIT involves language or other forms of communication intended to appear authoritative or persuasive without regard to its actual truth or logical consistency.[/quote]
- ta8645 2y ago
  That is an emotionally manipulative definition and anthropomorphizes the LLMs. They don't "intend" anything, they're not trying to trick you, or sound persuasive.
  - Freak_NL 2y ago
    They address this in lesson 2: > According to philosopher Harry Frankfurt, a liar knows the truth and is trying to lead us in the opposite direction. > A bullshitter either doesn't know the truth, or doesn't care. They are just trying to be persuasive. Being persuasive (i.e., churn out convincing prose) is how LLMs were designed to be.
    - ta8645 2y ago
      > Being persuasive (i.e., churn out convincing prose) is how LLMs were designed to be. No. They were designed to churn out accurate prose that accurately reflects their model of reality. They're just imperfect. You're being cynical and emotional to use the term bullshit. And again, it anthropomorphizes the LLM, it implies agency.
      - dijksterhuis 2y ago
        > that accurately reflects their model of reality. you are also seemingly anthropomorphising the technology by assigning to it some concept of having a “model of reality”. LLM systems output an inference of the next most likely token, given: the input prompt, the model weights and the previously output token [0]. that is all. no models of reality involved. “it” doesn’t “know” or “model” anything about “reality”. the systems are just a fancy probability maths pipelines. probably generally best to avoid using the word “they” in these discussions. the english language sucks sometimes. :shrug: [0]: yes i know it is a bit more complicated than that.
tomjen3 2y ago
I really hope that was intentional and the full effect of that naming choice was known beforehand, because I have already written the whole thing off, and I don't believe I'm the only one.
ArchitectAnon 2y ago
I wrote a post below about how AI hallucinated a whole regulation that didn’t exist which has been flagged for some reason. I have colleagues who have had arguments with clients who have asked AI questions about planning law and been given bullshit which they then insist is true and they can’t understand why thier architects won’t submit the appeal that they’re asking for. I think we’re in an era where any text, true or not, is so easy to generate and disseminate that the status of the written word is reduced to the standard of the gossip that used to be our main source of information before the printing press was invented. Now half the internet is AI generated bullshit as well.
- throwup238 2y ago
  It was flagged because it’s a copy paste of the same comment you made ten days ago.
  - ArchitectAnon 2y ago
    Oh right, sorry. Should I have linked to it? It seemed pretty relevant to this discussion and most of it was quoting something so I reused a lot of it. Didn't realise that was taboo.
foobiekr 2y ago
they are just bullshit machines. bullshitters can cite wikipedia and are still bullshitters who are bullshitting.

anon373839 2y ago

“Bullshit” actually means something: > In philosophy and psychology of cognition, the term "bullshit" is sometimes used to specifically refer to statements produced without particular concern for truth, clarity, or meaning, distinguishing "bullshit" from a deliberate, manipulative lie intended to subvert the truth. https://en.m.wikipedia.org/wiki/Bullshit https://en.m.wikipedia.org/wiki/Bullshit It’s really an ideal term to describe what LLMs do.

HPsquared 2y ago
I prefer "waffle" https://en.m.wikipedia.org/wiki/Waffle_(speech) https://en.m.wikipedia.org/wiki/Waffle_(speech) "Waffle machines" is even kind of funny.
- deleted 2y ago
  [deleted]
- fzzzy 2y ago
  Waffle machines is way better. Love it. Thanks.
- m0llusk 2y ago
  Makes me imagine coming up to the hindquarters of a bull with a waffle machine on an extension cord.
- burch45 2y ago
  Waffle means something very different though in the U.S., to “flip-flop” on a position. Not hold it for any fundamental reasons. But I don’t think you can say that an LLM holds a position whatsoever. Also, https://en.m.wikipedia.org/wiki/On_Bullshit https://en.m.wikipedia.org/wiki/On_Bullshit the essay originally popularizing the definition of bullshit considered here. Note the references to LLM output at he bottom.
TeMPOraL 2y ago
> It’s really an ideal term to describe what LLMs do. Only when you relax it to the point it also describes what most people do. To be specific: if by "truth" you mean objective, verifiable truth, and by "caring about truth" you mean caring about objective, empirically verifiable evidence, then by far most people are mostly only ever trying to be persuasive.
- anon373839 2y ago
  I hope you don't actually mean that. It's a cynical, sad, and - quite frankly - false thing to believe.
  - TeMPOraL 2y ago
    Why? I don't have a problem recognizing that "being convincing" to the right people is a very good way of learning and verifying knowledge about objective reality. Just because it's by proxy, doesn't mean it doesn't work. This is how every one of us learns most things, and what's really sad to me is deluding yourself into thinking we're more special than that.
    - anon373839 2y ago
      Honestly, it just sounds to me like you’ve come up with some axioms in your mind about the “fundamental nature” of humanity - and then granted yourself the luxury of certainty about them. That, with a dash of misanthropy, is something I’ve been seeing more and more on HN these days.

BrenBarn 2y ago

Yes, they are.

immibis 2y ago

It is literally a machine that says what you want to hear. Was anyone in this thread around for Talk to Transformer? Remember the unicorns demo? They may have now figured out how to wrangle the technology so it's more likely to spit facts, but it's still the same thing underneath.

aqueueaqueue 2y ago

I have been thinking about this and I have an idea for local LLM I want to try. It is based on the assumption LLMs make mistakes (they will!) but be confident about it. Use a 1B model and ask it about a city for fun mistakes in an otherwise impressive array of facts. Then you will see how bullshitty LLMs are. Anyway the idea is to constrain the LLM and a language understanding and to choose a constrained response. For example give it the capability to read out something from my calendar. Using the logits it gets to choose what that calendar item is. But then regular old code does the lookup and writes a canned sentence saying "At 6pm today you have a meeting titled $title". This way, my meeting schedule won't make my LLM talk like a pirate. This massively culls what the LLM can do but what it does do, is like a search and so just like Google (before gen AI!) you get a result but you can judge it as a human.

IceDane 2y ago
This is basically how a lot of it works already. What you are describing is more or less just structured output and tool usage.
- aqueueaqueue 2y ago
  Yes! Thanks for the terms. I guess I am saying restrict it to that. Probably how Siri etc. works. This gives a low bullshit usage pattern.

mvellandi 2y ago

Or are they modern day oracles? The full title is inclusive and invites consideration. Besides, how many different types of models are there? How are they used? For freely available ones made accessible to the general public via interfaces, when were they last updated? Do the interface implementors even care or was it a simple project to make money via ads and microtransactions? This is a new area of media literacy and requires critical thinking. Nonetheless, I do acknowledge the value of RAG-based approaches that attempt to qualify their reasoning through provided sources.

CharlesW 2y ago
> Or are they modern day oracles? Neither, which is probably why the parent commenter considers it provocative. "A bullshitter either doesn't know the truth, or doesn't care. They are just trying to be persuasive." In any case, this kind of anthropomorphization is definitely bullshit.

CyMonk 2y ago

they define the term bullshit in lesson 2: [quote]BULLSHIT involves language or other forms of communication intended to appear authoritative or persuasive without regard to its actual truth or logical consistency.[/quote]

ta8645 2y ago
That is an emotionally manipulative definition and anthropomorphizes the LLMs. They don't "intend" anything, they're not trying to trick you, or sound persuasive.
- Freak_NL 2y ago
  They address this in lesson 2: > According to philosopher Harry Frankfurt, a liar knows the truth and is trying to lead us in the opposite direction. > A bullshitter either doesn't know the truth, or doesn't care. They are just trying to be persuasive. Being persuasive (i.e., churn out convincing prose) is how LLMs were designed to be.
  - ta8645 2y ago
    > Being persuasive (i.e., churn out convincing prose) is how LLMs were designed to be. No. They were designed to churn out accurate prose that accurately reflects their model of reality. They're just imperfect. You're being cynical and emotional to use the term bullshit. And again, it anthropomorphizes the LLM, it implies agency.
    - dijksterhuis 2y ago
      > that accurately reflects their model of reality. you are also seemingly anthropomorphising the technology by assigning to it some concept of having a “model of reality”. LLM systems output an inference of the next most likely token, given: the input prompt, the model weights and the previously output token [0]. that is all. no models of reality involved. “it” doesn’t “know” or “model” anything about “reality”. the systems are just a fancy probability maths pipelines. probably generally best to avoid using the word “they” in these discussions. the english language sucks sometimes. :shrug: [0]: yes i know it is a bit more complicated than that.

tomjen3 2y ago

I really hope that was intentional and the full effect of that naming choice was known beforehand, because I have already written the whole thing off, and I don't believe I'm the only one.

ArchitectAnon 2y ago

I wrote a post below about how AI hallucinated a whole regulation that didn’t exist which has been flagged for some reason. I have colleagues who have had arguments with clients who have asked AI questions about planning law and been given bullshit which they then insist is true and they can’t understand why thier architects won’t submit the appeal that they’re asking for. I think we’re in an era where any text, true or not, is so easy to generate and disseminate that the status of the written word is reduced to the standard of the gossip that used to be our main source of information before the printing press was invented. Now half the internet is AI generated bullshit as well.

throwup238 2y ago
It was flagged because it’s a copy paste of the same comment you made ten days ago.
- ArchitectAnon 2y ago
  Oh right, sorry. Should I have linked to it? It seemed pretty relevant to this discussion and most of it was quoting something so I reused a lot of it. Didn't realise that was taboo.

foobiekr 2y ago

they are just bullshit machines. bullshitters can cite wikipedia and are still bullshitters who are bullshitting.