4 ms·

>you can be intelligent only if you fully imitate biological processes It is not that. It is about having an understanding of how it is trained. For example, i

by qsera 8mo ago

>you can be intelligent only if you fully imitate biological processes It is not that. It is about having an understanding of how it is trained. For example, if it was trained on ideas, instead of words, then it would be closer to intelligent behavior. Someone will say that during training it builds ideas and concepts, but that is just a name that we give for the internal representation that results from training and is not actual ideas and concepts. When it learns about the word "car", it does not actually understand it as a concept, but just as a word and how it can relate to other words. This enables it to generate words that include "car" that are consistent, projecting an appearance of intelligence. It is hard to propose a test for this, because it will become the next target for the AI companies to optimize for, and maybe the next model will pass it.

red75prime 8mo ago
The latest models are mostly LMMs (large multimodal models). If a model builds an internal representation that integrates all the modalities we are dealing with (robotics even provides tactile inputs), it becomes harder and harder to imagine why those representations should be qualitatively different.
- qsera 8mo ago
  It can't, simply because the textual description of a concept is different from the concept itself.
  - red75prime 8mo ago
    Obviously, a concept (which is an abstraction in more ways than one) is different from a textual representation. But LLMs don't operate on the textual description of a concept when they are doing their thing. A textual description (which is associated with other modalities in the training data) serves as an input format. LLMs perform non-linear transformations of points in their latent space. These transformations and representations are useful not only for generating text but also for controlling robots, for example (see VLAs in robotics).
    - qsera 8mo ago
      > don't operate on the textual description of a concept when they are doing their thing. It could be mapping the text to some other internal representation with connections to mappings from some other text/tokens. But it does not stop text from being the ground truth. It has nothing else going on! The "hallucination" behavior alone should be enough to reject any claims that these are at least minimally similar to animal intelligence.
      - red75prime 8mo ago
        The internal representation happen to be useful not only for outputting text. What does it mean from your standpoint?
        qsera 8mo ago
        I didn't understand. Can you clarify?
        red75prime 8mo ago
        If LLMs' internal representations are essentially one-to-one mappings of input texts with no additional structure, how can those representations be useful for tasks like object manipulation in robotics? How is transfer learning possible when non-textual training data enhances performance on textual tasks?
        qsera 8mo ago
        I didn't mean it is a one to one mapping from tokens. But instead it might be mapping a corpus of input text to some points in some multi dimensional space, (just like the input data a linear regression), then then it just extends the line further across that space to get the output. >How is transfer learning possible when non-textual training data enhances performance on textual tasks? If non-textual training data can be mapped to the same multi-dimensional space ( by using them alongside textual data during training or something like that), then shouldn't it be possible to do what you describe?

by qsera 8mo ago

red75prime 8mo ago

The latest models are mostly LMMs (large multimodal models). If a model builds an internal representation that integrates all the modalities we are dealing with (robotics even provides tactile inputs), it becomes harder and harder to imagine why those representations should be qualitatively different.

qsera 8mo ago
It can't, simply because the textual description of a concept is different from the concept itself.
- red75prime 8mo ago
  Obviously, a concept (which is an abstraction in more ways than one) is different from a textual representation. But LLMs don't operate on the textual description of a concept when they are doing their thing. A textual description (which is associated with other modalities in the training data) serves as an input format. LLMs perform non-linear transformations of points in their latent space. These transformations and representations are useful not only for generating text but also for controlling robots, for example (see VLAs in robotics).
  - qsera 8mo ago
    > don't operate on the textual description of a concept when they are doing their thing. It could be mapping the text to some other internal representation with connections to mappings from some other text/tokens. But it does not stop text from being the ground truth. It has nothing else going on! The "hallucination" behavior alone should be enough to reject any claims that these are at least minimally similar to animal intelligence.
    - red75prime 8mo ago
      The internal representation happen to be useful not only for outputting text. What does it mean from your standpoint?
      - qsera 8mo ago
        I didn't understand. Can you clarify?
        red75prime 8mo ago
        If LLMs' internal representations are essentially one-to-one mappings of input texts with no additional structure, how can those representations be useful for tasks like object manipulation in robotics? How is transfer learning possible when non-textual training data enhances performance on textual tasks?
        qsera 8mo ago
        I didn't mean it is a one to one mapping from tokens. But instead it might be mapping a corpus of input text to some points in some multi dimensional space, (just like the input data a linear regression), then then it just extends the line further across that space to get the output. >How is transfer learning possible when non-textual training data enhances performance on textual tasks? If non-textual training data can be mapped to the same multi-dimensional space ( by using them alongside textual data during training or something like that), then shouldn't it be possible to do what you describe?