4 ms·

The very fact that it is able to search within a meaning-space demonstrates that it understands semantics, to some extent. Philosophically, that is profound, fo

The very fact that it is able to search within a meaning-space demonstrates that it understands semantics, to some extent. Philosophically, that is profound, for something that is just one big matrix multiplication. Drawing connections between things in meaning-space is surely a facet of intelligence.

deleted 1mo ago
[deleted]
bayindirh 1mo ago
It’s not intelligence if you are the one who gives the correlations to the model in the pre-training. It’s Word2Vec, applied. Model doesn’t learn anything. You embed these correlations and build it from there. It just searches the space. As my AI professor said in the first lecture: “All AI is advanced search”.
- deleted 1mo ago
  [deleted]
- pyridines 1mo ago
  Okay, I guess you're right that its ability to do this is just correlational, which doesn't imply it has any understanding. However, you have to conclude that some tasks which we used to believe required intelligence don't actually require any, which is disconcerting.
  - bayindirh 1mo ago
    No, what I would say is the tasks which are handled in a passable manner by LLMs can be mathematically modeled with some reasonable accuracy. Many things are predicted by models in our planet. From weather to production and material science. Building the model needs intelligence, running the model does not. The person who came up with the formulae for CFD was intelligent. The computer running the model is not. Same for LLMs, chess engines, engine ECUs and financial prediction systems. Again, for the example’s sake; the person who came up with an algorithm is intelligent. The model mixing its training data to emit something similar is not.
    - hombre_fatal 1mo ago
      This starts to feel like you're defining the word intelligence out of any meaning and out of any way we apply that word. So when LLMs can do all human knowledge work, and do it better than humans, we'll be in the mines listening to you go on about how it's actually just autocomplete or just math, a distinction that apparently means nothing.
      - bayindirh 1mo ago
        > This starts to feel like you're defining the word intelligence out of any meaning and out of any way we apply that word. No. > So when LLMs can do all human knowledge work, and do it better than humans, we'll be in the mines listening to you go on about how it's actually just autocomplete or just math, a distinction that apparently means nothing. With a big "if" attached to it. People were saying "computers will program themselves in the near future" for, checks notes, 24 years now, as far as I'm aware. We're constantly building new knowledge and understanding things better than olden days. These models just compress our knowledge and light the blind corners we can't see well. I don't say they are useless, but I say that these things are overhyped. All they can do is regurgitate human knowledge packed into them and highlight some long-distance correlations between items, which is useful in itself, but it can't jump to somewhere where it's not present its training data, but that's something humans and only humans can do.
        layla5alive 1mo ago
        Locked in a dark room with no sensory organs, humans couldn't do that. Most of what you said reads to me as denial. An unconscious unintelligent but persistent trial and error process created us. We created LLMs. LLMs may create the next thing before we do - hard to say. They don't have all the cognitive tools we have yet, but they still outperform in some areas. As the cognitive playing field levels, I expect you will come to eat your words..
        ojo-rojo 1mo ago
        > it can't jump to somewhere where it's not present its training data That sounds like something that can be engineered, can't it? In other words, we can identify limitations in current transformer-based architectures, and we can also build new architectures over time.
- _ifton 1mo ago
  They perform tasks too. They execute functions. This has real world implications beyond search.

deleted 1mo ago

[deleted]

bayindirh 1mo ago

It’s not intelligence if you are the one who gives the correlations to the model in the pre-training. It’s Word2Vec, applied. Model doesn’t learn anything. You embed these correlations and build it from there. It just searches the space. As my AI professor said in the first lecture: “All AI is advanced search”.

deleted 1mo ago
[deleted]
pyridines 1mo ago
Okay, I guess you're right that its ability to do this is just correlational, which doesn't imply it has any understanding. However, you have to conclude that some tasks which we used to believe required intelligence don't actually require any, which is disconcerting.
- bayindirh 1mo ago
  No, what I would say is the tasks which are handled in a passable manner by LLMs can be mathematically modeled with some reasonable accuracy. Many things are predicted by models in our planet. From weather to production and material science. Building the model needs intelligence, running the model does not. The person who came up with the formulae for CFD was intelligent. The computer running the model is not. Same for LLMs, chess engines, engine ECUs and financial prediction systems. Again, for the example’s sake; the person who came up with an algorithm is intelligent. The model mixing its training data to emit something similar is not.
  - hombre_fatal 1mo ago
    This starts to feel like you're defining the word intelligence out of any meaning and out of any way we apply that word. So when LLMs can do all human knowledge work, and do it better than humans, we'll be in the mines listening to you go on about how it's actually just autocomplete or just math, a distinction that apparently means nothing.
    - bayindirh 1mo ago
      > This starts to feel like you're defining the word intelligence out of any meaning and out of any way we apply that word. No. > So when LLMs can do all human knowledge work, and do it better than humans, we'll be in the mines listening to you go on about how it's actually just autocomplete or just math, a distinction that apparently means nothing. With a big "if" attached to it. People were saying "computers will program themselves in the near future" for, checks notes, 24 years now, as far as I'm aware. We're constantly building new knowledge and understanding things better than olden days. These models just compress our knowledge and light the blind corners we can't see well. I don't say they are useless, but I say that these things are overhyped. All they can do is regurgitate human knowledge packed into them and highlight some long-distance correlations between items, which is useful in itself, but it can't jump to somewhere where it's not present its training data, but that's something humans and only humans can do.
      - layla5alive 1mo ago
        Locked in a dark room with no sensory organs, humans couldn't do that. Most of what you said reads to me as denial. An unconscious unintelligent but persistent trial and error process created us. We created LLMs. LLMs may create the next thing before we do - hard to say. They don't have all the cognitive tools we have yet, but they still outperform in some areas. As the cognitive playing field levels, I expect you will come to eat your words..
      - ojo-rojo 1mo ago
        > it can't jump to somewhere where it's not present its training data That sounds like something that can be engineered, can't it? In other words, we can identify limitations in current transformer-based architectures, and we can also build new architectures over time.
_ifton 1mo ago
They perform tasks too. They execute functions. This has real world implications beyond search.