7 ms·

I'm of the opinion that the current architectures are fundamentally ridden with "hallucinations" that will severely limit their practical usage (including very

I'm of the opinion that the current architectures are fundamentally ridden with "hallucinations" that will severely limit their practical usage (including very much what the hype thinks they could do). But this article puts an impossible limit to what it is to "not-hallucinate". It essentially restates well known fundamental limitations of formal systems and mechanistic computation and then presents the trivial result that LLMs also share these limitations. Unless some dualism or speculative supercomputational quantum stuff is invoked, this holds very much to humans too.

diggan 2y ago
> fundamentally ridden with "hallucinations" that will severely limit their practical usage On the other hand, a LLM that got rid of "hallucinations" is basically just a thing that copy-paste at that point. The interesting properties from LLMs comes from the fact that it can kind of make things up but still make them believable.
- jampekka 2y ago
  As per this article, even copy-paste hallucinate e.g. because there are no infinite datasets.
- SoftTalker 2y ago
  > can kind of make things up but still make them believable This is the definition of a bullshitter, by the way.
  - Kerb_ 2y ago
    Or, you know, fiction writer. Some of us like little stories.
    - _heimdall 2y ago
      Its important that anyone reading fiction knows its fiction. Without that we can run into a situation where a fictional story on the radio convinces the public that we are under alien attack in what could be called a war of the worlds.
      - Kerb_ 2y ago
        Yes! Or a situation where the public was not under any misconception at all, where there were multiple intermissions throughout the play calling attention to the fact that it was a work of fiction, but someone made up the idea of mass panic because it made for a good story that sold a lot of newspapers! I would say that it's the responsibility of the reader to read any content in the context of which it's published. For example, to expect a radio play to be fictional as the vast majority of listeners did.
- Too 2y ago
  Training with inappropriate data will make copy-paste equally bad with hallucinating. For example, fiction or sarcasm may be taken out of context and used as a serious output. What's missing is sanity-checking the sources and the output.
User23 2y ago
C.S. Peirce, who is known for characterizing abductive reasoning and had a considerable on John Sowa’s old school AI work, had an interesting take on this. I can’t fully do it justice, but essentially he held that both matter and mind are real, but aren’t dual. Rather, there is a smooth and continuous transition between the two. However, whatever the nature of mind and matter really is, we have convincing evidence of human beings creating meaning in symbols by a process Peirce called semiosis. We lack a properly formally described semiotic, although much interesting mathematical applied philosophy has been done in the space (and frankly a ton of bullshit in the academy calls itself semiotic too). Until we can do that, we will probably have great difficulty producing an automaton that can perform semiosis. So, for now, there certainly remains a qualitative difference between the capabilities of humans and LLMs.
- wrs 2y ago
  I don’t know the technical philosophy terms for this, but my simplistic way of thinking about it is that when I’m “seriously” talking (not just emitting thoughtless cliché phrases), I’m talking about something. And this is observable because sometimes I have an idea that I have trouble expressing in words, where I know that the words I’m saying are not properly expressing the idea that I have. (I mean — that’s happening right now!) I don’t see how that could ever happen for an LLM, because all it does is express things in words, and all it knows is the words that people expressed things with. We know for sure, that’s just what the code does; there’s no question about the underlying mechanism, like there is with humans.
  - User23 2y ago
    > I have an idea that I have trouble expressing in words, where I know that the words I’m saying are not properly expressing the idea that I have. (I mean — that’s happening right now!) That's a quality insight. Which, come to think of it, is an interestingly constructed word given what you just said.
  - gradus_ad 2y ago
    Producing text is only the visible end product. The LLM is doing a whole lot behind the scenes, which is conceivably analogous to the thought space from which our own words flow.
o11c 2y ago
What impresses me is frankly how bad it is. I can't claim to have tried every model out there, but most models very quickly fail when asked to do something along the lines of "describe the interaction of 3 entities." They can usually handle 2 (up to the point where they inevitably start talking in circles - often repeating entire chunks verbatim in many models), but 3 seems utterly beyond them. LLMs might have a role in the field of "burn money to generate usually-wrong ideas that are cheap enough to check in case there's actually a good one" though.

diggan 2y ago

> fundamentally ridden with "hallucinations" that will severely limit their practical usage On the other hand, a LLM that got rid of "hallucinations" is basically just a thing that copy-paste at that point. The interesting properties from LLMs comes from the fact that it can kind of make things up but still make them believable.

jampekka 2y ago
As per this article, even copy-paste hallucinate e.g. because there are no infinite datasets.
SoftTalker 2y ago
> can kind of make things up but still make them believable This is the definition of a bullshitter, by the way.
- Kerb_ 2y ago
  Or, you know, fiction writer. Some of us like little stories.
  - _heimdall 2y ago
    Its important that anyone reading fiction knows its fiction. Without that we can run into a situation where a fictional story on the radio convinces the public that we are under alien attack in what could be called a war of the worlds.
    - Kerb_ 2y ago
      Yes! Or a situation where the public was not under any misconception at all, where there were multiple intermissions throughout the play calling attention to the fact that it was a work of fiction, but someone made up the idea of mass panic because it made for a good story that sold a lot of newspapers! I would say that it's the responsibility of the reader to read any content in the context of which it's published. For example, to expect a radio play to be fictional as the vast majority of listeners did.
Too 2y ago
Training with inappropriate data will make copy-paste equally bad with hallucinating. For example, fiction or sarcasm may be taken out of context and used as a serious output. What's missing is sanity-checking the sources and the output.

User23 2y ago

C.S. Peirce, who is known for characterizing abductive reasoning and had a considerable on John Sowa’s old school AI work, had an interesting take on this. I can’t fully do it justice, but essentially he held that both matter and mind are real, but aren’t dual. Rather, there is a smooth and continuous transition between the two. However, whatever the nature of mind and matter really is, we have convincing evidence of human beings creating meaning in symbols by a process Peirce called semiosis. We lack a properly formally described semiotic, although much interesting mathematical applied philosophy has been done in the space (and frankly a ton of bullshit in the academy calls itself semiotic too). Until we can do that, we will probably have great difficulty producing an automaton that can perform semiosis. So, for now, there certainly remains a qualitative difference between the capabilities of humans and LLMs.

wrs 2y ago
I don’t know the technical philosophy terms for this, but my simplistic way of thinking about it is that when I’m “seriously” talking (not just emitting thoughtless cliché phrases), I’m talking about something. And this is observable because sometimes I have an idea that I have trouble expressing in words, where I know that the words I’m saying are not properly expressing the idea that I have. (I mean — that’s happening right now!) I don’t see how that could ever happen for an LLM, because all it does is express things in words, and all it knows is the words that people expressed things with. We know for sure, that’s just what the code does; there’s no question about the underlying mechanism, like there is with humans.
- User23 2y ago
  > I have an idea that I have trouble expressing in words, where I know that the words I’m saying are not properly expressing the idea that I have. (I mean — that’s happening right now!) That's a quality insight. Which, come to think of it, is an interestingly constructed word given what you just said.
- gradus_ad 2y ago
  Producing text is only the visible end product. The LLM is doing a whole lot behind the scenes, which is conceivably analogous to the thought space from which our own words flow.

o11c 2y ago

What impresses me is frankly how bad it is. I can't claim to have tried every model out there, but most models very quickly fail when asked to do something along the lines of "describe the interaction of 3 entities." They can usually handle 2 (up to the point where they inevitably start talking in circles - often repeating entire chunks verbatim in many models), but 3 seems utterly beyond them. LLMs might have a role in the field of "burn money to generate usually-wrong ideas that are cheap enough to check in case there's actually a good one" though.