5 ms·

In real world conversations, people are constantly saying "I don't know"; but that doesn't really happen online. If you're on reddit or stack overflow or hacker

In real world conversations, people are constantly saying "I don't know"; but that doesn't really happen online. If you're on reddit or stack overflow or hacker news and you see a question you don't know the answer to, you normally just don't say anything. If LLMs are being trained on conversations pulled from the internet then they're missing out on a ton of uncertain responses. Maybe LLMs don't truly "understand" questions, but they're good at looking like they understand questions. If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well.

username332211 3y ago
If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set? Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? When the LLM answers "I don't know.", this could be a hallucination just as easily as anything else.
- deleted 3y ago
  [deleted]
- caditinpiscinam 3y ago
  > Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? I don't know :) Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velocity of an unladen swallow, and 99 of them told me "I have no idea" then I'd be pretty skeptical of the one ornithologist who did give me an answer, even if they were very confident.
  - ta8645 3y ago
    I think the best response is to steal the joke and repeat it without comment. "Eleven meters per second." Full stop. It's humorous, and any reasonable interlocutor understands not to take it seriously. Of course, there are more serious questions that demand more serious answers. LLMs will eventually need to be able to understand the current context and assess the appropriate level of confidence required in any answer.
  - username332211 3y ago
    The thing is, the usefulness of a question answering system is in answering questions people don't generally know. We don't need an answering system for things that are common knowledge. And it's not uncommon that certain knowledge would be, well uncommon even among experts. Experts specialize. Since the usefulness of ornithological examples is getting exhausted, let's say one out of a hundred lawyers works in bankruptcy. If you ask a million lawyers about the provisions of 11 USC § 1129 and only ten thousand know the answer, is the answer untrustworthy, just because bankruptcy lawyers are far rarer than civil and criminal lawyers?
    - caditinpiscinam 3y ago
      I agree that without deeper reasoning skills it's not obvious how a model should handle this sort of situation. In the end, I'm not saying that fixing the hallucination issue is easy, but rather that hallucination might be better understood as a property of the training data everyone is using, rather than the models themselves.
- patmcc 3y ago
  Right, but "I don't know" is a pretty safe hallucination (if it is one). My main worry about hallucinations is it means I absolutely can't rely on the output for anything important. If I ask what the safe dose for Tylenol for an infant is, the answer needs to be either correct or "I don't know". It's not acceptable for it to hallucinate 10x the safe dose.
  - username332211 3y ago
    The thing is, if you answer "I don't know" based on statistics, you end up creating a sliding scale of sorts. You get some measure of an increase in safety, but the model is less useful. Currently,we have models that make stuff up when they don't know the answer. On the other end, we'd have a model that's refuses to answer any question that's not common knowledge. It'll be safe (though it can never be completely safe), but essentially useless. I suspect it'll be impossible to make a completely trustworthy and useful model unless it somehow has a concept of it's own knowledge. And can you have a concept of one's knowledge if you lack a concept of self?
fl7305 3y ago
If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not.
- beardedwizard 3y ago
  This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data.
  - fl7305 3y ago
    >>>> They cannot say "I dont know" >>> If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well. >> (me) If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not. > This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data. First of all, my claim was in response to "They cannot say 'I dont know'" and "perhaps they'd be better at expressing uncertainty". ChatGPT can say "I don't know" if you ask it to. Regarding whether LLMs are lookup tables, I responded to that in more detail elsewhere under this post: https://news.ycombinator.com/item?id=39501611 https://news.ycombinator.com/item?id=39501611
  - naasking 3y ago
    And your post was the most probable output of your mind process given your experiences. The only self-evident difference is the richness of your experience as compared to LLMs.
    - beardedwizard 3y ago
      No the self evidence difference is that the brain is equipped with many more models than simply language. Language is one of the ways we express the composite output of many models, emotion being a key other which has no need for language to exist. This is why it is a fallacy to think an LLM contains anything other than the textual descriptions of our higher level thinking, and why LLM alone will only ever parrot intelligence.

username332211 3y ago

If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set? Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? When the LLM answers "I don't know.", this could be a hallucination just as easily as anything else.

deleted 3y ago
[deleted]
caditinpiscinam 3y ago
> Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? I don't know :) Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velocity of an unladen swallow, and 99 of them told me "I have no idea" then I'd be pretty skeptical of the one ornithologist who did give me an answer, even if they were very confident.
- ta8645 3y ago
  I think the best response is to steal the joke and repeat it without comment. "Eleven meters per second." Full stop. It's humorous, and any reasonable interlocutor understands not to take it seriously. Of course, there are more serious questions that demand more serious answers. LLMs will eventually need to be able to understand the current context and assess the appropriate level of confidence required in any answer.
- username332211 3y ago
  The thing is, the usefulness of a question answering system is in answering questions people don't generally know. We don't need an answering system for things that are common knowledge. And it's not uncommon that certain knowledge would be, well uncommon even among experts. Experts specialize. Since the usefulness of ornithological examples is getting exhausted, let's say one out of a hundred lawyers works in bankruptcy. If you ask a million lawyers about the provisions of 11 USC § 1129 and only ten thousand know the answer, is the answer untrustworthy, just because bankruptcy lawyers are far rarer than civil and criminal lawyers?
  - caditinpiscinam 3y ago
    I agree that without deeper reasoning skills it's not obvious how a model should handle this sort of situation. In the end, I'm not saying that fixing the hallucination issue is easy, but rather that hallucination might be better understood as a property of the training data everyone is using, rather than the models themselves.
patmcc 3y ago
Right, but "I don't know" is a pretty safe hallucination (if it is one). My main worry about hallucinations is it means I absolutely can't rely on the output for anything important. If I ask what the safe dose for Tylenol for an infant is, the answer needs to be either correct or "I don't know". It's not acceptable for it to hallucinate 10x the safe dose.
- username332211 3y ago
  The thing is, if you answer "I don't know" based on statistics, you end up creating a sliding scale of sorts. You get some measure of an increase in safety, but the model is less useful. Currently,we have models that make stuff up when they don't know the answer. On the other end, we'd have a model that's refuses to answer any question that's not common knowledge. It'll be safe (though it can never be completely safe), but essentially useless. I suspect it'll be impossible to make a completely trustworthy and useful model unless it somehow has a concept of it's own knowledge. And can you have a concept of one's knowledge if you lack a concept of self?

fl7305 3y ago

If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not.

beardedwizard 3y ago
This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data.
- fl7305 3y ago
  >>>> They cannot say "I dont know" >>> If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well. >> (me) If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not. > This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data. First of all, my claim was in response to "They cannot say 'I dont know'" and "perhaps they'd be better at expressing uncertainty". ChatGPT can say "I don't know" if you ask it to. Regarding whether LLMs are lookup tables, I responded to that in more detail elsewhere under this post: https://news.ycombinator.com/item?id=39501611 https://news.ycombinator.com/item?id=39501611
- naasking 3y ago
  And your post was the most probable output of your mind process given your experiences. The only self-evident difference is the richness of your experience as compared to LLMs.
  - beardedwizard 3y ago
    No the self evidence difference is that the brain is equipped with many more models than simply language. Language is one of the ways we express the composite output of many models, emotion being a key other which has no need for language to exist. This is why it is a fallacy to think an LLM contains anything other than the textual descriptions of our higher level thinking, and why LLM alone will only ever parrot intelligence.