5 ms·

there is no channel for uncertainty. LLMs of this type will just start making up shit when they dont know something. because they simply generate the most prob

there is no channel for uncertainty. LLMs of this type will just start making up shit when they dont know something. because they simply generate the most probable next token based on previous x tokens. this is not fixable. this alone makes these LLMs practically unusable in vast majority of real-world applications where you would otherwise imagine this tech to be used.

greyman 4y ago
What if LLM knowledge will expand over time to be sufficient for certain real-world application?
- Jeff_Brown 4y ago
  It already is: translation.
jazzyjackson 4y ago
yea its a simulator of human text on the internet for instance, your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? These things didn't have any of these emergent capabilities one year ago, why are you so sure you understand their nature one year from now?
- lamp987 4y ago
  "your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? " no its because GPT is based on transformers.
  - coldtea 4y ago
    and you aren't? Aren't you just a function of your input and memories (stuff you've read, sensory input) as run through/managed by some neural network? What makes you think the rest isn't just emergent properties? And what makes you think you can't hook up the LLM with some algorithms or layers that handle some of the rest behavior of what your brain does?
    - pixl97 4y ago
      Yep, the idea of grounding seems interesting to me. Everything in a LLM is just a statistical dream at this point with no 'reality basis' at this point. I wonder if it's possible to give the language model grounding points of things that are real and building a truth model from that.
    - lamp987 4y ago
      no, our brains arent based on transformers. and the issue of lost uncertainty is inherrent to this, yes. to fix this, a new type of llm would have to be invented. this particular branch of development may very well be a dead end.
antibasilisk 4y ago
The reason they seem to make things up is because they have no way to verify anything, they can only speak of things in relation to other things, but they have no epistemic framework. This is very much a fixable problem that augmentation with logic engines and a way to prioritise truth-claims could go some ways towards solving.
- Jeff_Brown 4y ago
  My memory could be improved by connecting my brain to an external hard drive. Wiring them together, alas, is not just hard; we have absolutely no idea how.
  - antibasilisk 4y ago
    We do have some idea how, most people just don't really want to deal with the nightmare of being augmented and the life changing consequences that come with it, on top of the risk.
    - Jeff_Brown 4y ago
      Really?! Have we done it in mice?
catskul2 4y ago
It's not clear why this would be a fundamental limit rather than a design flaw that will eventually be solved.
- Jeff_Brown 4y ago
  It might get solved but we have no idea how. There's no (readable) db of facts and no (visible) logical processing to improve on.
  - catskul2 4y ago
    To me "we have no idea how" != "this is not fixable" (and even "we have no idea how" even seems like a strong statement.) Perhaps it's because I'm ignorant about the inner workings, but calling the problem "unfixable" so early in the evolution of LLMs seems like foolish certainty.
    - Jeff_Brown 4y ago
      I pretty much agree. That wasn't me who called it unfixable. (This is a UI issue with HN that keeps coming up for me -- people being mistaken for the OP.)
LeanderK 4y ago
this is absolutely not a fundamental limit but simply a hard challenge. Approaches exist and it is an active field of research where we do make progress.
- Jeff_Brown 4y ago
  I would disagree with both of you. It's an open question whether LLMs can be made reliable.
  - LeanderK 4y ago
    fair. It's not proven that a solution exists for our models but I don't see much that leads me to believe it's impossible. I know GPT is not reliable but there's also really not much done to improve reliability. Its open research but certainly interesting. Most approaches I know are developed on way smaller datasets, models and usually in a computer vision context.

greyman 4y ago

What if LLM knowledge will expand over time to be sufficient for certain real-world application?

Jeff_Brown 4y ago
It already is: translation.

jazzyjackson 4y ago

yea its a simulator of human text on the internet for instance, your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? These things didn't have any of these emergent capabilities one year ago, why are you so sure you understand their nature one year from now?

lamp987 4y ago
"your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? " no its because GPT is based on transformers.
- coldtea 4y ago
  and you aren't? Aren't you just a function of your input and memories (stuff you've read, sensory input) as run through/managed by some neural network? What makes you think the rest isn't just emergent properties? And what makes you think you can't hook up the LLM with some algorithms or layers that handle some of the rest behavior of what your brain does?
  - pixl97 4y ago
    Yep, the idea of grounding seems interesting to me. Everything in a LLM is just a statistical dream at this point with no 'reality basis' at this point. I wonder if it's possible to give the language model grounding points of things that are real and building a truth model from that.
  - lamp987 4y ago
    no, our brains arent based on transformers. and the issue of lost uncertainty is inherrent to this, yes. to fix this, a new type of llm would have to be invented. this particular branch of development may very well be a dead end.

antibasilisk 4y ago

The reason they seem to make things up is because they have no way to verify anything, they can only speak of things in relation to other things, but they have no epistemic framework. This is very much a fixable problem that augmentation with logic engines and a way to prioritise truth-claims could go some ways towards solving.

Jeff_Brown 4y ago
My memory could be improved by connecting my brain to an external hard drive. Wiring them together, alas, is not just hard; we have absolutely no idea how.
- antibasilisk 4y ago
  We do have some idea how, most people just don't really want to deal with the nightmare of being augmented and the life changing consequences that come with it, on top of the risk.
  - Jeff_Brown 4y ago
    Really?! Have we done it in mice?

catskul2 4y ago

It's not clear why this would be a fundamental limit rather than a design flaw that will eventually be solved.

Jeff_Brown 4y ago
It might get solved but we have no idea how. There's no (readable) db of facts and no (visible) logical processing to improve on.
- catskul2 4y ago
  To me "we have no idea how" != "this is not fixable" (and even "we have no idea how" even seems like a strong statement.) Perhaps it's because I'm ignorant about the inner workings, but calling the problem "unfixable" so early in the evolution of LLMs seems like foolish certainty.
  - Jeff_Brown 4y ago
    I pretty much agree. That wasn't me who called it unfixable. (This is a UI issue with HN that keeps coming up for me -- people being mistaken for the OP.)

LeanderK 4y ago

this is absolutely not a fundamental limit but simply a hard challenge. Approaches exist and it is an active field of research where we do make progress.

Jeff_Brown 4y ago
I would disagree with both of you. It's an open question whether LLMs can be made reliable.
- LeanderK 4y ago
  fair. It's not proven that a solution exists for our models but I don't see much that leads me to believe it's impossible. I know GPT is not reliable but there's also really not much done to improve reliability. Its open research but certainly interesting. Most approaches I know are developed on way smaller datasets, models and usually in a computer vision context.