6 ms·

> What this tells us for AI is that we need something else besides LLMs Not to over-hype LLMs, but I don't see why this results says this. AI doesn't need to d

> What this tells us for AI is that we need something else besides LLMs Not to over-hype LLMs, but I don't see why this results says this. AI doesn't need to do things the same way as evolved intelligence has.

weard_beard 2y ago
To a point. If you drill down this far into the fundamentals of cognition you begin to define it. Otherwise you may as well call a cantaloupe sentient
- jebarker 2y ago
  I don't think anyone defines AI as "doing the thing that biological brains do" though, we define it in terms of capabilities of the system.
  - weard_beard 2y ago
    I think if you gave it the same biological inputs as a biological brain you would quickly see the lack of capabilities in any man made system.
    - Dylan16807 2y ago
      Okay, but does that help us reach any meaningful conclusions? For example, okay some AI system doesn't have the capabilities of an auditory cortex or somatosensory cortex. Is there a reason for me to think it needs that?
      - weard_beard 2y ago
        Name a creature on earth without one. Imagine trying to limit, control, or explain a being without familiar cognitive structures. Is there a reason to care about such unfamiliar modalities of cognition?
        Dylan16807 2y ago
        > Name a creature on earth without one. Anything that doesn't have a spine, I'm pretty sure. Also if we look at just auditory, tons of creatures are deaf and don't need that. > Imagine trying to limit, control, or explain a being without familiar cognitive structures. I don't see why any of that that affects whether it's intelligent.
        weard_beard 2y ago
        Agreed: Perhaps we aught to be studying cognition of creatures without spines before we claim to replicate or understand cognition of creatures with them. Presumably they have some sort biological input processing or sensory inputs. They don't eat data.
heavyset_go 2y ago
It doesn't need to, but evolved intelligence is the only intelligence we know of. Similar reason we look for markers of Earth-based life on alien planets: it's the only example we've got of it existing.
zbyforgotp 2y ago
Ok, but at least it suggests that this other thing might be more efficient in some ways.
awongh 2y ago
One reason might that LLMs are successful because of the architecture, but also, just as importantly because they can be trained over a volume and diversity of human thought that’s encapsulated in language (that is on the internet). Where are we going to find the equivalent data set that will train this other kind of thinking? Open AI O1 seems to be trained on mostly synthetic data, but it makes intuitive sense that LLMs work so well because we had the data lying around already.
- jebarker 2y ago
  I think the data is way more important for the success of LLMs than the architecture although I do think there's something important in the GPT architecture in particular. See this talk for why: [1] Warning, watch out for waving hands: The way I see it is that cognition involves forming an abstract representation of the world and then reasoning about that representation. It seems obvious that non-human animals do this without language. So it seems likely that humans do too and then language is layered on top as a turbo boost. However, it also seems plausible that you could build an abstract representation of the world through studying a vast amount of human language and that'll be a good approximation of the real-world too and furthermore it seems possible that reasoning about that abstract representation can take place in the depths of the layers of a large transformer. So it's not clear to me that we're limited by the data we have or necessarily need a different type of data to build a general AI although that'll likely help build a better world model. It's also not clear that an LLM is incapable of the type of reasoning that animals apply to their abstract world representations. [1] https://youtu.be/yBL7J0kgldU?si=38Jjw_dgxCxhiu7R https://youtu.be/yBL7J0kgldU?si=38Jjw_dgxCxhiu7R
  - tsimionescu 2y ago
    > However, it also seems plausible that you could build an abstract representation of the world through studying a vast amount of human language and that'll be a good approximation of the real-world too and furthermore it seems possible that reasoning about that abstract representation can take place in the depths of the layers of a large transformer. While I agree this is possible, I don't see why you'd think it's likely. I would instead say that I think it's unlikely. Human communication relies on many assumptions of a shared model of the world that are rarely if ever discussed explicitly, and without which certain concepts or at least phrases become ambiguous or hard to understand.
uoaei 2y ago
in the high entropy world we have, we are forced to assume that the first thing that arises as a stable pattern is inevitably the most likely, and the most likely to work. there is no other pragmatic conclusion to draw. for more, see "Assembly Theory"
numpad0 2y ago
Title doesn't mean bullet trains can't fly, but do imply what call flights could be more than moving fast, and effects of wings might be worth discussing.
lanstin 2y ago
Language models would seem to be exquisitely tied to the way that evolved intelligence has formulated its society and training. An Ab Initio AGI would maybe be free of our legacy, but LLMs certainly are not. I would expect a ship-like intelligence a la the Culture novels to have non-English based cognition. As far as we can tell, our own language generation is post-hoc explanation for thought more so than the embodiment of thought.

weard_beard 2y ago

To a point. If you drill down this far into the fundamentals of cognition you begin to define it. Otherwise you may as well call a cantaloupe sentient

jebarker 2y ago
I don't think anyone defines AI as "doing the thing that biological brains do" though, we define it in terms of capabilities of the system.
- weard_beard 2y ago
  I think if you gave it the same biological inputs as a biological brain you would quickly see the lack of capabilities in any man made system.
  - Dylan16807 2y ago
    Okay, but does that help us reach any meaningful conclusions? For example, okay some AI system doesn't have the capabilities of an auditory cortex or somatosensory cortex. Is there a reason for me to think it needs that?
    - weard_beard 2y ago
      Name a creature on earth without one. Imagine trying to limit, control, or explain a being without familiar cognitive structures. Is there a reason to care about such unfamiliar modalities of cognition?
      - Dylan16807 2y ago
        > Name a creature on earth without one. Anything that doesn't have a spine, I'm pretty sure. Also if we look at just auditory, tons of creatures are deaf and don't need that. > Imagine trying to limit, control, or explain a being without familiar cognitive structures. I don't see why any of that that affects whether it's intelligent.
        weard_beard 2y ago
        Agreed: Perhaps we aught to be studying cognition of creatures without spines before we claim to replicate or understand cognition of creatures with them. Presumably they have some sort biological input processing or sensory inputs. They don't eat data.

heavyset_go 2y ago

It doesn't need to, but evolved intelligence is the only intelligence we know of. Similar reason we look for markers of Earth-based life on alien planets: it's the only example we've got of it existing.

zbyforgotp 2y ago

Ok, but at least it suggests that this other thing might be more efficient in some ways.

awongh 2y ago

One reason might that LLMs are successful because of the architecture, but also, just as importantly because they can be trained over a volume and diversity of human thought that’s encapsulated in language (that is on the internet). Where are we going to find the equivalent data set that will train this other kind of thinking? Open AI O1 seems to be trained on mostly synthetic data, but it makes intuitive sense that LLMs work so well because we had the data lying around already.

jebarker 2y ago
I think the data is way more important for the success of LLMs than the architecture although I do think there's something important in the GPT architecture in particular. See this talk for why: [1] Warning, watch out for waving hands: The way I see it is that cognition involves forming an abstract representation of the world and then reasoning about that representation. It seems obvious that non-human animals do this without language. So it seems likely that humans do too and then language is layered on top as a turbo boost. However, it also seems plausible that you could build an abstract representation of the world through studying a vast amount of human language and that'll be a good approximation of the real-world too and furthermore it seems possible that reasoning about that abstract representation can take place in the depths of the layers of a large transformer. So it's not clear to me that we're limited by the data we have or necessarily need a different type of data to build a general AI although that'll likely help build a better world model. It's also not clear that an LLM is incapable of the type of reasoning that animals apply to their abstract world representations. [1] https://youtu.be/yBL7J0kgldU?si=38Jjw_dgxCxhiu7R https://youtu.be/yBL7J0kgldU?si=38Jjw_dgxCxhiu7R
- tsimionescu 2y ago
  > However, it also seems plausible that you could build an abstract representation of the world through studying a vast amount of human language and that'll be a good approximation of the real-world too and furthermore it seems possible that reasoning about that abstract representation can take place in the depths of the layers of a large transformer. While I agree this is possible, I don't see why you'd think it's likely. I would instead say that I think it's unlikely. Human communication relies on many assumptions of a shared model of the world that are rarely if ever discussed explicitly, and without which certain concepts or at least phrases become ambiguous or hard to understand.

uoaei 2y ago

in the high entropy world we have, we are forced to assume that the first thing that arises as a stable pattern is inevitably the most likely, and the most likely to work. there is no other pragmatic conclusion to draw. for more, see "Assembly Theory"

numpad0 2y ago

Title doesn't mean bullet trains can't fly, but do imply what call flights could be more than moving fast, and effects of wings might be worth discussing.

lanstin 2y ago

Language models would seem to be exquisitely tied to the way that evolved intelligence has formulated its society and training. An Ab Initio AGI would maybe be free of our legacy, but LLMs certainly are not. I would expect a ship-like intelligence a la the Culture novels to have non-English based cognition. As far as we can tell, our own language generation is post-hoc explanation for thought more so than the embodiment of thought.