10 ms·

Encyclopedia Britannica is also wrong in a reproducible and fixable way. And the input queries a finite set. It's output does not change due to random or arbit

Encyclopedia Britannica is also wrong in a reproducible and fixable way. And the input queries a finite set. It's output does not change due to random or arbitrary things. It is actually possible to verify. LLMs so far seem to be entirely unverifiable.

lukev 2y ago
They don’t just seem it. They are by design. We talk about models “hallucinating” but that’s us bringing an external value judgement after the fact. The actual process of token generation works precisely the same. It’d be more accurate to say that models always hallucinate.
- madeofpalk 2y ago
  Yes - this is what i've been saying all the time. The term 'hallucinations' is misleading because the whole point of LLMs is that they recombine all their inputs into something 'new'. They only ever hallucinate outputs - that's their whole point!
  - wizzwizz4 2y ago
    Into something probable. The models that underlie these chatbots are usually overfitted, so while they usually don't repeat their training data verbatim, they can.
- DougBTX 2y ago
  > The actual process of token generation works precisely the same I’d be wary of generalising it like that, it is like saying that all programs run on the same set of CPU instructions. NNs are function approximators, where the code is expressed in model weights rather than text, but that doesn’t make all functions the same.
  - lukev 2y ago
    You misunderstand. I mean that the model itself is doing exactly the same thing whether the output is a “hallucination “ or happens to be fact. There isn’t even a theoretical way to distinguish between the two cases based only on the information encoded in the model.
  - skydhash 2y ago
    > it is like saying that all programs run on the same set of CPU instructions Turing machine is the embodiment of all computer programs. And then you come across the halting problem. LLMs can probably generate all books in existence, but it can't apply judgement to it. Just like you need programmers to actually write the program and verify that it correctly solves the problem. Natural languages are more flexible. There are no functions libraries, or paradigms to ease writing. And the problem space can't be specified and usually relies on shared context. Even if we could have snippets of prompts to guide text generations, the result is not that valuable.
- intended 2y ago
  YES. Humans can hallucinate, its a deviation from what is observable reality. All the stress people are feeling with GenAI comes from the over anthropomorphisation of ... stats. Impressive syntatic ability is not equivalent to semantic capability.
  - tremon 2y ago
    The human definition of hallucination has to do with sensory experience, i.e. inputs. Saying that LLMs hallucinate means that we're ascribing them control over their inputs that they simply do not have -- by design. Or, in other words, if a chatbot really were hallucinating, it would probably start giving unprompted responses. > Humans can hallucinate, its a deviation from what is observable reality. What is "observable reality" then, for an LLM? Its training set?
ta8645 2y ago
> LLMs so far seem to be entirely unverifiable. I don't understand this complaint. Are they any less verifiable than a human?
- threeseed 2y ago
  I can ask a human to explain the steps they took to answer a question. I can ask a human a question 100 times and I don't get back 100 different answers. None of those applies to an LLM.
  - ta8645 2y ago
    You can ask an LLM to explain itself.. it will give you a logical stepwise progression from your question to its answer. It will often contain a mistake, but the same is true for a human. And if your LLM is giving you 100 different answers, then it has been configured to do so. Because instead, it could be configured to never vary at all. It could be 100% reproducible if so desired.
    - soloist11 2y ago
      [dead]
    - bluefirebrand 2y ago
      > it will give you a logical stepwise progression from your question to its answer. No, it will generate a new hallucination that might be a logical stepwise progression from the question you asked to the answer you gave, but it is not due to any actual internal reasoning being done by the LLM
      - ta8645 2y ago
        So what? You have no way to know for sure if the human you ask the same question, does either. The question that started this thread was related to verifiability. And i still think it is a spurious complaint, given that we have exactly the same limitations when dealing with any human agent.
somenameforme 2y ago
LLMs are completely deterministic even if that's kind of weird to state because they output things in terms of probabilities. But if you simply took the highest probability next word, you'd always yield the exact same output given the exact same input. Randomness is intentionally injected to make them seem less robotic through the 'temperature' parameter. Why it's not just called the rng factor is beyond me.
- dleeftink 2y ago
  Maybe some models can be deterministic at a point in time, but train it for another epoch with slight parameter changes and a revised corpus and determinism goes out the proverbial (sliding) window real quick. This is not unwanted per se, and the exact feedback loop that needs improving to better integrate new knowledge or revise knowledge artefacts incrementally/post-hoc.
  - Vetch 2y ago
    If you train it then it's no longer the same model. If I have f(x) = x + 1 and change it to f(x) = x + 1 + 1/1e9, it would not mean that `f` is not deterministic. The issue would be in whatever interface I was exposing the f's at.
    - jononor 2y ago
      But current models must be retrained to incorporate new information. Or to attempt to fix undesirable behavior. So just freezing it forever does not seem feasible. And because there is no way to predict what has changed - one has to verify everything all over again.
      - avar 2y ago
        Would you by extension argue that e.g. modern relational database aren't deterministic in their query execution? Their query plans tend to be chosen based on statistics about the tables they're executed against, and not just the query itself. I don't see how that's different than the LLM case, a lot of algorithms change as a function of the data they're processing.
ein0p 2y ago
LLMs are deterministic if you want them to be. If you eagerly argmax the output you will get the same sequence for the same prompt every time

lukev 2y ago

They don’t just seem it. They are by design. We talk about models “hallucinating” but that’s us bringing an external value judgement after the fact. The actual process of token generation works precisely the same. It’d be more accurate to say that models always hallucinate.

madeofpalk 2y ago
Yes - this is what i've been saying all the time. The term 'hallucinations' is misleading because the whole point of LLMs is that they recombine all their inputs into something 'new'. They only ever hallucinate outputs - that's their whole point!
- wizzwizz4 2y ago
  Into something probable. The models that underlie these chatbots are usually overfitted, so while they usually don't repeat their training data verbatim, they can.
DougBTX 2y ago
> The actual process of token generation works precisely the same I’d be wary of generalising it like that, it is like saying that all programs run on the same set of CPU instructions. NNs are function approximators, where the code is expressed in model weights rather than text, but that doesn’t make all functions the same.
- lukev 2y ago
  You misunderstand. I mean that the model itself is doing exactly the same thing whether the output is a “hallucination “ or happens to be fact. There isn’t even a theoretical way to distinguish between the two cases based only on the information encoded in the model.
- skydhash 2y ago
  > it is like saying that all programs run on the same set of CPU instructions Turing machine is the embodiment of all computer programs. And then you come across the halting problem. LLMs can probably generate all books in existence, but it can't apply judgement to it. Just like you need programmers to actually write the program and verify that it correctly solves the problem. Natural languages are more flexible. There are no functions libraries, or paradigms to ease writing. And the problem space can't be specified and usually relies on shared context. Even if we could have snippets of prompts to guide text generations, the result is not that valuable.
intended 2y ago
YES. Humans can hallucinate, its a deviation from what is observable reality. All the stress people are feeling with GenAI comes from the over anthropomorphisation of ... stats. Impressive syntatic ability is not equivalent to semantic capability.
- tremon 2y ago
  The human definition of hallucination has to do with sensory experience, i.e. inputs. Saying that LLMs hallucinate means that we're ascribing them control over their inputs that they simply do not have -- by design. Or, in other words, if a chatbot really were hallucinating, it would probably start giving unprompted responses. > Humans can hallucinate, its a deviation from what is observable reality. What is "observable reality" then, for an LLM? Its training set?

ta8645 2y ago

> LLMs so far seem to be entirely unverifiable. I don't understand this complaint. Are they any less verifiable than a human?

threeseed 2y ago
I can ask a human to explain the steps they took to answer a question. I can ask a human a question 100 times and I don't get back 100 different answers. None of those applies to an LLM.
- ta8645 2y ago
  You can ask an LLM to explain itself.. it will give you a logical stepwise progression from your question to its answer. It will often contain a mistake, but the same is true for a human. And if your LLM is giving you 100 different answers, then it has been configured to do so. Because instead, it could be configured to never vary at all. It could be 100% reproducible if so desired.
  - soloist11 2y ago
    [dead]
  - bluefirebrand 2y ago
    > it will give you a logical stepwise progression from your question to its answer. No, it will generate a new hallucination that might be a logical stepwise progression from the question you asked to the answer you gave, but it is not due to any actual internal reasoning being done by the LLM
    - ta8645 2y ago
      So what? You have no way to know for sure if the human you ask the same question, does either. The question that started this thread was related to verifiability. And i still think it is a spurious complaint, given that we have exactly the same limitations when dealing with any human agent.

somenameforme 2y ago

LLMs are completely deterministic even if that's kind of weird to state because they output things in terms of probabilities. But if you simply took the highest probability next word, you'd always yield the exact same output given the exact same input. Randomness is intentionally injected to make them seem less robotic through the 'temperature' parameter. Why it's not just called the rng factor is beyond me.

dleeftink 2y ago
Maybe some models can be deterministic at a point in time, but train it for another epoch with slight parameter changes and a revised corpus and determinism goes out the proverbial (sliding) window real quick. This is not unwanted per se, and the exact feedback loop that needs improving to better integrate new knowledge or revise knowledge artefacts incrementally/post-hoc.
- Vetch 2y ago
  If you train it then it's no longer the same model. If I have f(x) = x + 1 and change it to f(x) = x + 1 + 1/1e9, it would not mean that `f` is not deterministic. The issue would be in whatever interface I was exposing the f's at.
  - jononor 2y ago
    But current models must be retrained to incorporate new information. Or to attempt to fix undesirable behavior. So just freezing it forever does not seem feasible. And because there is no way to predict what has changed - one has to verify everything all over again.
    - avar 2y ago
      Would you by extension argue that e.g. modern relational database aren't deterministic in their query execution? Their query plans tend to be chosen based on statistics about the tables they're executed against, and not just the query itself. I don't see how that's different than the LLM case, a lot of algorithms change as a function of the data they're processing.

ein0p 2y ago

LLMs are deterministic if you want them to be. If you eagerly argmax the output you will get the same sequence for the same prompt every time