3 ms·

> If the LLMs had said something to the effect of "I don't know, I wasn't trained to answer <question type> about <question topic>" ... This is so very much th

> If the LLMs had said something to the effect of "I don't know, I wasn't trained to answer <question type> about <question topic>" ... This is so very much the answer I want from an LLM every single time it's not reasonably certain about an answer to a query. Not "hallucinate" a plausible sounding answer and then confidently spew lies at me as if it's gospel truth. I can accept "I don't know" from a human just fine, so I can damn sure accept it from a machine made by humans; but I'm far less tolerant of lies from a machine than I am from humans. Humans will lie for a great many reasons, a fair few of which are easily enough forgivable / understandable. A machine will generally "lie" for only a comparatively very few possible reasons (a physical flaw or defect in the machine; faulty data, be it purposeful or accidental; a human designed it to be untruthful, etc.), most of which are just plain largely unacceptable on multiple levels. Even better than "I don't know" would be my fifth grade teacher's favorite answer ("way back in ye golden olden days") in situations where he didn't know the answer: "I don't know, but let's find out together." and then the research would proceed apace. An LLM should be capable of such quite easily one would think. They make great "research assistants" when trained on a relevant data set and guided properly with a well crafted system prompt, and they're centered around / trained upon human language so should be able to guide a human to available resources with pretty near zero hassle. :)

lasereyes136 2y ago
I think part of the point of the article is that LLMs don't lie because they are designed to just give you the next work based on making a credible sounding sentence or sequence of sentences. Expecting it to do more is an expectations problem based on the hype around GenAI. I don't think we have the correct word for what LLMs do but lie and hallucinations are not really correct.
- codewench 2y ago
  > I don't think we have the correct word for what LLMs do but lie and hallucinations are not really correct. I believe 'bullshit' is accurate, as in "The chatbot didn't know the answer, so it started bullshitting".
- Tagbert 2y ago
  I think hallucination is pretty close. It represents what happens when you give an answer based on what you think you remember even if that memory is not correct. How many people would agree that P.T. Barnum said “There’s a sucker born every minute”? That would be a hallucination. The quote is from Adam Forepaugh.
  - unaindz 2y ago
    The best argument I have found against using lie or hallucination for describing LLM's actions is that it humanizes them to people who don't know the inner workings of LLMs. Saying they lie gives intent which is pretty bad but even hallucination humanizes them unnecessarily. Bullshitting seems the best word to describe it but even then intent can be assumed when there isn't any.
    - nicklecompte 2y ago
      I said "confabulate" in my original post. "Confabulation" is a neurological symptom commonly seen in dementia patients, where a person isn't telling the truth because of errors in memory formation/recall or some other non-psychological problem in the brain. In particular, people who confabulate aren't aware their words are false and therefore it doesn't make sense to say that they're "lying." Likewise it's a problem with memory, not perception, so "hallucination" doesn't work either. "Confabulation" still isn't great because humans confabulate with non-verbal memories and then express those confabulations in words; human confabulation mostly affects biographical memory, not subject matter knowledge. But considering how weird it is to even be talking about "memory" with a being that isn't aware of the passage of time, I think "confabulate" is the best option short of inventing a brand new word.
      - haliyat 2y ago
        “Bullshit” is the _perfect_ term. Philosopher Harry Frankfurter wrote a book called On Bullshit where he defines the term as speech or writing intended to persuade without regard for the truth. This is _exactly_ what LLMs do. They produce text that tries to reproduce the average properties of texts in their training data and the user experiences the encoded in their RLHF training. None of that has anything to do with the truth. At best you could say they are engineered to try to give the users what they want (eg. what the engineers building these systems think we want), which is, again, a common motive of bullshitters.
- HarHarVeryFunny 2y ago
  Saying "I don't know" doesn't require too much of a change. This isn't a different mode of operation where it's introspecting about its own knowledge - it's just the best continuation prediction in a context where the person/entity being questioned is not equipped to answer. LLMs create quite deep representations of the input on which they based their next word prediction (text continuation), and it has been proved that they already sometimes do know when something they are generating is low confidence or false, so maybe with appropriate training data they could better attend to this and predict "I don't know" or "I'm not sure". To improve the ability of LLMs to answer like this requires them to have a better idea of what is true or not. Humans do this by remembering where they learnt something: was it first hand experience, or from a text book or trusted friend, or from a less trustworthy source. LLMs ability to discern the truth could be boosted by giving them the sources of their training data, maybe together with a trustworthiness rating (although they may be able to learn that for themselves).
akomtu 2y ago
What does it even mean to lie for a text generator? It outputs the most probable continuation of the given input and that continuation is indeed the most probable in its training dataset. We don't say that DNA sequences are true or false.
- HarHarVeryFunny 2y ago
  Good prediction involves modelling the data generator, including hidden state such as level of knowledge or uncertainly, motivation, or tendency to lie. If you ask a question of someone/something who is ill equipped to answer, then (assuming you have not modeled them as inveterate bullshitter) a good predicted response is "I don't know". Deliberately lying due to a motivation to deceive is different from the default LLM mode of "just keep talking" bullshitting. The only "motivation" an LLM has is to predict next word, but if it knows that this requires lying then it will do so (e.g. give it a setup where it is someone motivated to lie).

lasereyes136 2y ago

I think part of the point of the article is that LLMs don't lie because they are designed to just give you the next work based on making a credible sounding sentence or sequence of sentences. Expecting it to do more is an expectations problem based on the hype around GenAI. I don't think we have the correct word for what LLMs do but lie and hallucinations are not really correct.

codewench 2y ago
> I don't think we have the correct word for what LLMs do but lie and hallucinations are not really correct. I believe 'bullshit' is accurate, as in "The chatbot didn't know the answer, so it started bullshitting".
Tagbert 2y ago
I think hallucination is pretty close. It represents what happens when you give an answer based on what you think you remember even if that memory is not correct. How many people would agree that P.T. Barnum said “There’s a sucker born every minute”? That would be a hallucination. The quote is from Adam Forepaugh.
- unaindz 2y ago
  The best argument I have found against using lie or hallucination for describing LLM's actions is that it humanizes them to people who don't know the inner workings of LLMs. Saying they lie gives intent which is pretty bad but even hallucination humanizes them unnecessarily. Bullshitting seems the best word to describe it but even then intent can be assumed when there isn't any.
  - nicklecompte 2y ago
    I said "confabulate" in my original post. "Confabulation" is a neurological symptom commonly seen in dementia patients, where a person isn't telling the truth because of errors in memory formation/recall or some other non-psychological problem in the brain. In particular, people who confabulate aren't aware their words are false and therefore it doesn't make sense to say that they're "lying." Likewise it's a problem with memory, not perception, so "hallucination" doesn't work either. "Confabulation" still isn't great because humans confabulate with non-verbal memories and then express those confabulations in words; human confabulation mostly affects biographical memory, not subject matter knowledge. But considering how weird it is to even be talking about "memory" with a being that isn't aware of the passage of time, I think "confabulate" is the best option short of inventing a brand new word.
    - haliyat 2y ago
      “Bullshit” is the _perfect_ term. Philosopher Harry Frankfurter wrote a book called On Bullshit where he defines the term as speech or writing intended to persuade without regard for the truth. This is _exactly_ what LLMs do. They produce text that tries to reproduce the average properties of texts in their training data and the user experiences the encoded in their RLHF training. None of that has anything to do with the truth. At best you could say they are engineered to try to give the users what they want (eg. what the engineers building these systems think we want), which is, again, a common motive of bullshitters.
HarHarVeryFunny 2y ago
Saying "I don't know" doesn't require too much of a change. This isn't a different mode of operation where it's introspecting about its own knowledge - it's just the best continuation prediction in a context where the person/entity being questioned is not equipped to answer. LLMs create quite deep representations of the input on which they based their next word prediction (text continuation), and it has been proved that they already sometimes do know when something they are generating is low confidence or false, so maybe with appropriate training data they could better attend to this and predict "I don't know" or "I'm not sure". To improve the ability of LLMs to answer like this requires them to have a better idea of what is true or not. Humans do this by remembering where they learnt something: was it first hand experience, or from a text book or trusted friend, or from a less trustworthy source. LLMs ability to discern the truth could be boosted by giving them the sources of their training data, maybe together with a trustworthiness rating (although they may be able to learn that for themselves).

akomtu 2y ago

What does it even mean to lie for a text generator? It outputs the most probable continuation of the given input and that continuation is indeed the most probable in its training dataset. We don't say that DNA sequences are true or false.

HarHarVeryFunny 2y ago
Good prediction involves modelling the data generator, including hidden state such as level of knowledge or uncertainly, motivation, or tendency to lie. If you ask a question of someone/something who is ill equipped to answer, then (assuming you have not modeled them as inveterate bullshitter) a good predicted response is "I don't know". Deliberately lying due to a motivation to deceive is different from the default LLM mode of "just keep talking" bullshitting. The only "motivation" an LLM has is to predict next word, but if it knows that this requires lying then it will do so (e.g. give it a setup where it is someone motivated to lie).