3 ms·

> By reaching into the voids of its embedding space and returning tokens related to nonexistent semantics. Or, if you like, "hallucinating". The hallucinations

by sudb 3mo ago

> By reaching into the voids of its embedding space and returning tokens related to nonexistent semantics. Or, if you like, "hallucinating". The hallucinations which are useful we might call "novel". This sounds to me like an admission that LLMs are not just doing a stochastic search, then. > close enough facsimile to intelligence What's the distinguishing criteria then? How can you tell the difference?

27183 3mo ago
I think we must be talking past eachother. I define stochastic search as a search process with randomness injected into it that can return the following things: - Something contained in the data set, not necessarily the same thing for every iteration of a given query - Something not contained the data set (hallucination), not necessarily the same thing for every iteration of a given query Does that clear it up? > What's the distinguishing criteria then? How can you tell the difference? All the ways they fail to exhibit intelligence. They can't learn. They can't adapt. They can't reason abstractly. They can't count. Etc...
- sudb 3mo ago
  Ah it seems you are a stochastic parrot believer (https://en.wikipedia.org/wiki/Stochastic_parrot https://en.wikipedia.org/wiki/Stochastic_parrot). What are your responses to the Expert Rebuttals section? I find the rebuttals pretty convincing - that there seems to be some emergent behaviour that is not simply just next-token-prediction, or that the ability to do accurate next-token-prediction requires something "extra" that LLMs have. > All the ways they fail to exhibit intelligence Another implicit admission that there _are_ ways that LLMs exhibit intelligence?
  - 27183 3mo ago
    > there seems to be some emergent behaviour that is not simply just next-token-prediction, or that the ability to do accurate next-token-prediction requires something "extra" that LLMs have. The next step then would be to design and conduct experiments that isolate this effect. Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining. Isolate it or distill it so it can be studied directly. Until then, it's easiest to dismiss it as imaginary.
    - sudb 3mo ago
      > Figure out how to make it happen reliably And you're happy that the replication of LLMs across many foundation model companies is insufficiently reliable? > just something you're imagining So the alternative explanation you're suggesting to emergent LLM behaviour is mass independently-corroborated human hallucination. Which is more likely? Also it really does seem like you've moved the goalposts a lot here without really giving me a substantive response.
      - 27183 3mo ago
        To say that LLMs' existence is evidence for emergent phenomena in LLMs is tautological. I'm merely suggesting if you want to make a claim about emergence it would be best, especially in absence of a convincing theory, to demonstrate it experimentally. Otherwise probably better not to claim it's actually happening.
        sudb 3mo ago
        > To say that LLMs' existence is evidence for emergent phenomena in LLMs is tautological. This is not at all what I was saying. I think you've already conceded that LLMs demonstrate emergent behaviour but you dismissed it as a "close enough facsimile to intelligence". I was saying that the emergent behaviour is reliably replicable, in response to your following statement: > Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining. I think there is real work underway in the area of interpretability. In the meantime, there appears to be plenty of empirical evidence for the claim that LLMs exhibit some sort "intelligence" in the enormous penetration that agentic coding has achieved in software development? Do you deny the usefulness of LLMs here, or are you going to assert that actually software development requires no intelligence of any sort?
        27183 3mo ago
        > I think you've already conceded that LLMs demonstrate emergent behaviour No. Please don't put words in my mouth. What I said is that an LLM compresses a bunch of information into a semantic embedding space and then does sort of a stochastic search in that embedding space. Any similarity to "intelligence" is accidental. You may look at the results of that process and "see" thinking or reasoning or something, but it ain't there. > "intelligence" ... agentic ... usefulness I don't think LLMs need to be intelligent to be (at least narrowly) useful. No more than random forests or genetic algorithms do at least. [edit] Look, this has devolved to the point where it's no longer productive to continue. If you're going to state things like this as fact, there's really nothing more I can do here: > emergent behaviour is reliably replicable Go collect your Nobel prize then! This is no longer a discussion grounded in reality.

by sudb 3mo ago

27183 3mo ago

I think we must be talking past eachother. I define stochastic search as a search process with randomness injected into it that can return the following things: - Something contained in the data set, not necessarily the same thing for every iteration of a given query - Something not contained the data set (hallucination), not necessarily the same thing for every iteration of a given query Does that clear it up? > What's the distinguishing criteria then? How can you tell the difference? All the ways they fail to exhibit intelligence. They can't learn. They can't adapt. They can't reason abstractly. They can't count. Etc...

sudb 3mo ago
Ah it seems you are a stochastic parrot believer (https://en.wikipedia.org/wiki/Stochastic_parrot https://en.wikipedia.org/wiki/Stochastic_parrot). What are your responses to the Expert Rebuttals section? I find the rebuttals pretty convincing - that there seems to be some emergent behaviour that is not simply just next-token-prediction, or that the ability to do accurate next-token-prediction requires something "extra" that LLMs have. > All the ways they fail to exhibit intelligence Another implicit admission that there _are_ ways that LLMs exhibit intelligence?
- 27183 3mo ago
  > there seems to be some emergent behaviour that is not simply just next-token-prediction, or that the ability to do accurate next-token-prediction requires something "extra" that LLMs have. The next step then would be to design and conduct experiments that isolate this effect. Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining. Isolate it or distill it so it can be studied directly. Until then, it's easiest to dismiss it as imaginary.
  - sudb 3mo ago
    > Figure out how to make it happen reliably And you're happy that the replication of LLMs across many foundation model companies is insufficiently reliable? > just something you're imagining So the alternative explanation you're suggesting to emergent LLM behaviour is mass independently-corroborated human hallucination. Which is more likely? Also it really does seem like you've moved the goalposts a lot here without really giving me a substantive response.
    - 27183 3mo ago
      To say that LLMs' existence is evidence for emergent phenomena in LLMs is tautological. I'm merely suggesting if you want to make a claim about emergence it would be best, especially in absence of a convincing theory, to demonstrate it experimentally. Otherwise probably better not to claim it's actually happening.
      - sudb 3mo ago
        > To say that LLMs' existence is evidence for emergent phenomena in LLMs is tautological. This is not at all what I was saying. I think you've already conceded that LLMs demonstrate emergent behaviour but you dismissed it as a "close enough facsimile to intelligence". I was saying that the emergent behaviour is reliably replicable, in response to your following statement: > Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining. I think there is real work underway in the area of interpretability. In the meantime, there appears to be plenty of empirical evidence for the claim that LLMs exhibit some sort "intelligence" in the enormous penetration that agentic coding has achieved in software development? Do you deny the usefulness of LLMs here, or are you going to assert that actually software development requires no intelligence of any sort?
        27183 3mo ago
        > I think you've already conceded that LLMs demonstrate emergent behaviour No. Please don't put words in my mouth. What I said is that an LLM compresses a bunch of information into a semantic embedding space and then does sort of a stochastic search in that embedding space. Any similarity to "intelligence" is accidental. You may look at the results of that process and "see" thinking or reasoning or something, but it ain't there. > "intelligence" ... agentic ... usefulness I don't think LLMs need to be intelligent to be (at least narrowly) useful. No more than random forests or genetic algorithms do at least. [edit] Look, this has devolved to the point where it's no longer productive to continue. If you're going to state things like this as fact, there's really nothing more I can do here: > emergent behaviour is reliably replicable Go collect your Nobel prize then! This is no longer a discussion grounded in reality.