4 ms·

Intelligence is more than just knowing the probabilistic relationship between every word.

Rhapso 2y ago
"Intelligence" is a continuous process. Without a continuous feedback loop, LLMs will never be more than a compression algorithm we bullied into being a chatbot. OpenAi as a mega-organism might be intelligent, but the LLMs definitely are not. The "compressed capture of semantic relationships" is a new thing we don't have a word for.
- thrance 2y ago
  Funnily enough, there is a mathematical link between data compression and AGI [1]. I believe a paper circulated some time ago that compared gpt2 to gzip, with interesting results. [1] https://en.wikipedia.org/wiki/AIXI https://en.wikipedia.org/wiki/AIXI
  - TeMPOraL 2y ago
    More than that, in general, understanding and compression seem to be fundamentally the same thing.
    - cootsnuck 2y ago
      I would say understanding requires compression not that it equates to it. Probably just semantics though.
3abiton 2y ago
It's part of the process, given that the "bigger picture" remains in context.
winwang 2y ago
Do you have strong evidence for this?
- aithrowawaycomm 2y ago
  Dogs are highly intelligent, and it makes no sense to say that they get their intelligence by calculating the probabilities between consecutive woofs.
  - mitthrowaway2 2y ago
    Would you say with equal confidence that they don't exemplify their intelligence by their ability to repeatedly select an often-successful next action from a set of possible next actions, based on a set of input observations? "Tokens" don't have to be words, or woofs...
    - aithrowawaycomm 2y ago
      It still doesn’t make sense for dogs. It might make some sense given a higher-level goal (hiding a toy under the bed)[1] but it doesn’t make much sense for selecting the goals (“I should hide this toy because the other dog keeps stealing it”). In building an AI dog it doesn’t work to elevate these higher-level goals into individual tokens because real dogs form goals dynamically according to their environment and the set is infinitely large. (Note that LLM agents also badly struggle with this; generating goals token-by-token means their goals have hallucinations.) [1] It still doesn’t make much sense to view this as a statistical process; dogs can generalize far better than transformers, as perhaps best seen with seeing-eye dogs. I believe dogs’ powers of causal reasoning exceed what is possible from mere surface statistics: e.g. they innately understand object permanence as puppies, whereas transformers still don’t understand it after viewing thousands of dogs’ lifetimes of experience.
      - mitthrowaway2 2y ago
        I've not been able to find any way to distinguish "mere surface statistics" from the deeper, richer, and more meaningful kind of something that it is meant to be contrasted with, except that "surface statistics" are un-compressed. For example, surface statistics might be the set of output measurements generated by a compact process, such as the positions of planets over time; knowing the laws of gravity means we can generate gigabytes of these statistics correctly and easily, which will accurately match future observations. But then going the other way, from statistics to a causal model, is just an inverse problem -- just like, say, going from a set of noisy magnetic field measurements at the boundary of a container to a pattern of electric current flow inside a volume, or going from planet positions to orbit shapes and periods to an inverse square law of gravity. Generating a compressed inverse model from surface statistics is exactly the sort of thing that deep learning has proven to be very good at. And by now we've seen no shortage of evidence that LLMs and other deep networks contain stateful world models, which is exactly what you'd expect, because for all their parameters, they aren't nearly big enough to contain an infinitesimal fraction of the statistics they were trained on. So I think it's overly dismissive to regard LLMs as mere surface statistics.
rsrsrs86 2y ago
Please spread the word that predicting the next one is not intelligence. It’s markov…
- esafak 2y ago
  It depends on how you predict. To predict better and better you need intelligence.
  - threeseed 2y ago
    Knowledge is distinct from intelligence. You can predict better and better with simply more knowledge i.e. data.
    - esafak 2y ago
      That only gets you as far as the frontier of knowledge. To go beyond you need intelligence.

"Intelligence" is a continuous process. Without a continuous feedback loop, LLMs will never be more than a compression algorithm we bullied into being a chatbot. OpenAi as a mega-organism might be intelligent, but the LLMs definitely are not. The "compressed capture of semantic relationships" is a new thing we don't have a word for.

thrance 2y ago
Funnily enough, there is a mathematical link between data compression and AGI [1]. I believe a paper circulated some time ago that compared gpt2 to gzip, with interesting results. [1] https://en.wikipedia.org/wiki/AIXI https://en.wikipedia.org/wiki/AIXI
- TeMPOraL 2y ago
  More than that, in general, understanding and compression seem to be fundamentally the same thing.
  - cootsnuck 2y ago
    I would say understanding requires compression not that it equates to it. Probably just semantics though.

3abiton 2y ago

It's part of the process, given that the "bigger picture" remains in context.

winwang 2y ago

Do you have strong evidence for this?

aithrowawaycomm 2y ago
Dogs are highly intelligent, and it makes no sense to say that they get their intelligence by calculating the probabilities between consecutive woofs.
- mitthrowaway2 2y ago
  Would you say with equal confidence that they don't exemplify their intelligence by their ability to repeatedly select an often-successful next action from a set of possible next actions, based on a set of input observations? "Tokens" don't have to be words, or woofs...
  - aithrowawaycomm 2y ago
    It still doesn’t make sense for dogs. It might make some sense given a higher-level goal (hiding a toy under the bed)[1] but it doesn’t make much sense for selecting the goals (“I should hide this toy because the other dog keeps stealing it”). In building an AI dog it doesn’t work to elevate these higher-level goals into individual tokens because real dogs form goals dynamically according to their environment and the set is infinitely large. (Note that LLM agents also badly struggle with this; generating goals token-by-token means their goals have hallucinations.) [1] It still doesn’t make much sense to view this as a statistical process; dogs can generalize far better than transformers, as perhaps best seen with seeing-eye dogs. I believe dogs’ powers of causal reasoning exceed what is possible from mere surface statistics: e.g. they innately understand object permanence as puppies, whereas transformers still don’t understand it after viewing thousands of dogs’ lifetimes of experience.
    - mitthrowaway2 2y ago
      I've not been able to find any way to distinguish "mere surface statistics" from the deeper, richer, and more meaningful kind of something that it is meant to be contrasted with, except that "surface statistics" are un-compressed. For example, surface statistics might be the set of output measurements generated by a compact process, such as the positions of planets over time; knowing the laws of gravity means we can generate gigabytes of these statistics correctly and easily, which will accurately match future observations. But then going the other way, from statistics to a causal model, is just an inverse problem -- just like, say, going from a set of noisy magnetic field measurements at the boundary of a container to a pattern of electric current flow inside a volume, or going from planet positions to orbit shapes and periods to an inverse square law of gravity. Generating a compressed inverse model from surface statistics is exactly the sort of thing that deep learning has proven to be very good at. And by now we've seen no shortage of evidence that LLMs and other deep networks contain stateful world models, which is exactly what you'd expect, because for all their parameters, they aren't nearly big enough to contain an infinitesimal fraction of the statistics they were trained on. So I think it's overly dismissive to regard LLMs as mere surface statistics.

rsrsrs86 2y ago

Please spread the word that predicting the next one is not intelligence. It’s markov…

esafak 2y ago
It depends on how you predict. To predict better and better you need intelligence.
- threeseed 2y ago
  Knowledge is distinct from intelligence. You can predict better and better with simply more knowledge i.e. data.
  - esafak 2y ago
    That only gets you as far as the frontier of knowledge. To go beyond you need intelligence.