4 ms·

Yes: An LLM isnt a model of human thinking. An LLM is an attempt to build a simulation of human communication. An LLM is to language what a forecast is to wea

Yes: An LLM isnt a model of human thinking. An LLM is an attempt to build a simulation of human communication. An LLM is to language what a forecast is to weather. No amount of weather data is actually going to turn that simulation into snow, no amount of LLM data is going to create AGI. That having been said, better models (smaller, more flexible ones) are going to result in a LOT of practical uses that have the potential to make our day to day lives easier (think digital personal assistant that has current knowledge).

choeger 3y ago
Great comment. Just one thought: Language, unlike weather, is meta-circular. All we know about specific words or sentences is again encoded in words and sentences. So the embedding encodes a subset of human knowledge. Hence, a LLM is predicting not only language but language with some sort of meaning.
- zer00eyz 3y ago
  That re-embeding is also encoded in weather. It is why perfect forecasting is impossible, why we talk about the butterfly effect. The "hallucination problem" is simply the tyranny of Lorenz... one is not sure if a starting state will have a good outcome or swing wildly. Some good weather models are based on re-runing with tweaks to starting params, and then things that end up out of bounds can get tossed. Its harder to know when a result is out of bounds for an LLM, and we dont have the ability to run every request 100 times through various models to get an "average" output yet... However some of the reuse of layers does emulate this to an extent....
red75prime 3y ago
Ugh. Really? Those "simulated water isn't wet"(when applied to cognition) "arguments" were punched so many times it even hurts to look at them.
- zer00eyz 3y ago
  No simulated water isnt wet. But an LLM isn't even trying to simulate cognition. It's a model that is predicting language. It has all the problems of a predictive model... the "hallucination" problem is just the tyranny of Lorenz.
  - adrianN 3y ago
    We don't really know what "cognition" is, so it's hard to tell whether a system is doing it.
  - lostmsu 3y ago
    This is plain wrong due to mixing of concepts. Language is technically something from Chomsky hierarchy. Predicting language is being able to tell if input is valid or invalid. LLMs do that, but they also build a statistical model across all valid inputs, and that is not just the language.
    - zer00eyz 3y ago
      >> Predicting language is being able to tell if input is valid or invalid. If this were the case then the hallucination problem would be solvable. That hallucination problem is not only going to be hard to detect in any meaningful way but it's going to be harder to eliminate. The very nature of LLM (mixing in noise aka temperature) means that they always risk going off the rails. This is the same thing Lorenz discovered in modeling weather...
      - lostmsu 3y ago
        I don't think that "hallucination problem" is a problem at all worth addressing separately from just building bigger/better models that do the same thing. Because 1) it is present in humans, 2) it is clear bigger models have less of it than smaller models. If at scale nothing changes LLMs will eventually just hallucinate less than humans.

choeger 3y ago

Great comment. Just one thought: Language, unlike weather, is meta-circular. All we know about specific words or sentences is again encoded in words and sentences. So the embedding encodes a subset of human knowledge. Hence, a LLM is predicting not only language but language with some sort of meaning.

zer00eyz 3y ago
That re-embeding is also encoded in weather. It is why perfect forecasting is impossible, why we talk about the butterfly effect. The "hallucination problem" is simply the tyranny of Lorenz... one is not sure if a starting state will have a good outcome or swing wildly. Some good weather models are based on re-runing with tweaks to starting params, and then things that end up out of bounds can get tossed. Its harder to know when a result is out of bounds for an LLM, and we dont have the ability to run every request 100 times through various models to get an "average" output yet... However some of the reuse of layers does emulate this to an extent....

red75prime 3y ago

Ugh. Really? Those "simulated water isn't wet"(when applied to cognition) "arguments" were punched so many times it even hurts to look at them.

zer00eyz 3y ago
No simulated water isnt wet. But an LLM isn't even trying to simulate cognition. It's a model that is predicting language. It has all the problems of a predictive model... the "hallucination" problem is just the tyranny of Lorenz.
- adrianN 3y ago
  We don't really know what "cognition" is, so it's hard to tell whether a system is doing it.
- lostmsu 3y ago
  This is plain wrong due to mixing of concepts. Language is technically something from Chomsky hierarchy. Predicting language is being able to tell if input is valid or invalid. LLMs do that, but they also build a statistical model across all valid inputs, and that is not just the language.
  - zer00eyz 3y ago
    >> Predicting language is being able to tell if input is valid or invalid. If this were the case then the hallucination problem would be solvable. That hallucination problem is not only going to be hard to detect in any meaningful way but it's going to be harder to eliminate. The very nature of LLM (mixing in noise aka temperature) means that they always risk going off the rails. This is the same thing Lorenz discovered in modeling weather...
    - lostmsu 3y ago
      I don't think that "hallucination problem" is a problem at all worth addressing separately from just building bigger/better models that do the same thing. Because 1) it is present in humans, 2) it is clear bigger models have less of it than smaller models. If at scale nothing changes LLMs will eventually just hallucinate less than humans.