9 ms·

It's not combining an LLM with data/memory recollection and infinite agency that will lead to leaps in cognition. Complex systems still can do that and still no

It's not combining an LLM with data/memory recollection and infinite agency that will lead to leaps in cognition. Complex systems still can do that and still not have self-awareness / cognition. We will need to take some more fundamental leaps to synthesize memory, agency and prediction to operate within a single system where self-awareness resides in the entity and not in the building blocks. LLM's represent a building block towards that cognitive unit, but the real deal is still some ways away

danryan 3y ago
Bingo.
strikelaserclaw 3y ago
Although i fundamentally agree with you, i will say that it is impossible to know which "tricks" will lead to a leap in cognition, even looking evolutionarily, the difference in genome between a monkey and human is very small percentage wise, yet the effects of that small difference are stark.
- unshavedyak 3y ago
  I'm always on the position that at some point (i make no prediction of when) "AI" will still be labeled as not-intelligent, but to external observers it won't be possible to know. Ie we will be able to describe how a function of `add(1,2)` is most clearly definitely not intelligent. We will understand the internal makeup of these neurons better, the whole system will be deterministic and understood - just a very large `add(1,2)` .. so it's definitely not intelligent, of course. Yet it will still produce output that is fully and truly turing-proof. If you let it run, it retains, stores, "behaves", etc. It's clearly just `add(1,2)` in a loop after all.. we will understand it so well that it's obviously just a trick of our mind that it seems real. The conclusion in that space will not be one of the machine .. but of us. If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us? The existential crisis will be neat.
  - c_crank 3y ago
    >If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us? Less than people think. We know that being a biological system differentiates us. If intelligent systems tend to converge where language is concerned, so be it. It doesn't mean the AI warrants any special consideration. It's still a tool.
    - mercurialsolo 3y ago
      While life is typically associated with biological systems, intelligence doesn't necessarily need proof of life. A hive, swarm or network can be intelligent as long as it shows self-awareness and has agency.
      - c_crank 3y ago
        I'm not disputing that. I just think that intelligence or 'sapience' is not something that requires reverence on our part.
        TeMPOraL 3y ago
        What is then, if anything? Definitely not "being a biological system" - that seems much more boring than intelligence, and the line between "biological" and "non-biological" is somewhat arbitrary and increasingly blurry anyway.
        c_crank 3y ago
        It is a quality that makes something good at drawing pictures and writing smart sound replies to prompts. And translating.
        jacobr1 3y ago
        I'm increasingly thinking the game-theoretic emergent properties of social interaction as described in papers like https://desirism.com/morality-from-the-ground-up https://desirism.com/morality-from-the-ground-up provides a good basis for thinking about this. The distinguishing factor then becomes something like, the agent is is participant in the moral-sphere by having "enough" properties to do so. It still leaves the details of that gradient a bit undefined, but at least specifies the basis upon which to make a judgment.
ImHereToVote 3y ago
Why not use less emotionally charged words? Goal to action mapping for instance. How far away are we from a general goal to action mapper that can outcompete a human being on a given goal.
indigo945 3y ago
I am an absolute layman when it comes to AI topics, but I have been wondering about this lately. A lot of the hopes in regard to AGI in LLMs seem to focus on just making the models bigger, or feeding them even more information. On the other hand, some researchers in the AGI space seem to move in a completely different direction, trying to use neural networks to simulate different parts of a brain, moving away from language altogether. But what I'm wondering is this: might it be a more productive avenue to consider an LLM not as an entire mind, but as a single part of an hypothetical artifical psyche, which excels at some of the overarching tasks of consciousness (such as acting within a cultural context and epistemological framework), but can delegate other aspects to more fitting subsystems? Could it be combined with other networks optimized for different tasks? Potentially, this could just be done by having inputs from other parts of this artificial psyche fed in as prompts, translated by an encoder/decoder of the likes of CLIP, i.e. a decoder that can translate ideas between different liminal spaces. The work that I keep remembering in this context is Jaynes' Origin of Consciousness in the Breakdown of the Bicameral Mind [1], which I stumbled upon here on HN [2] [3]. Jaynes believes that consciousness is not an innate ability, but a cultural achievement - in order to become conscious, one first has to learn (through others) to conceptualize consciousness, and to think of oneself as a thinking being. He insists that consciousness is not a necessity for human agency, and that humanity has been building civilizations and using technologies long before humans discovered (or achieved) consciousness. Jaynes puts forward the idea of an ancient, non-conscious human mind based on, upon other things, "verbal hallucinations" - thoughts and memories that involuntarily enter as associations, but whose origins are interpreted by the mind to not be within itself, but somewhere outside of it, such as in a divine monologue; thought, to the pre-conscious mind, is the incessant blabbering of the Gods. It seems that LLMs would excel in the role of this "camera" of the mind - as the source of divine monologue and divine instructions, that can then be compared with memories, interpreted and executed by more specialized subsystems. Might it be feasible to give up the plan to make AGI self-aware altogether and instead build an AGI powered by a GPT-4 ascended to divinity, just as ignorant of its own existence as ever? [1]: https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in_the_Breakdown_of_the_Bicameral_Mind https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in... [2]: https://news.ycombinator.com/item?id=27917316 https://news.ycombinator.com/item?id=27917316 [3]: https://news.ycombinator.com/item?id=27923444 https://news.ycombinator.com/item?id=27923444
empath-nirvana 3y ago
what is the 'real deal', exactly? I feel like people have this vague idea in their head of what everyone is trying to build, but don't really have it defined in a concrete way.
- mcpackieh 3y ago
  The fundamental issue is that all the talk of "self-awareness", "cognition", "souls", whatever you want to call it, is all beating around the bush to describe what we presumably each subjectively experience in our own minds but which cannot be empirically tested for in others. We generally assume that other people experience the same, but cannot empirically prove it and some people speculate that others might lack it (P-zombies, "NPCs", etc). And then when it comes to animals, a huge portion of the population refuse to believe animals can possess it. I think the majority belief is that some animals definitely have it (dolphins) and some animals probably do not (jellyfish? corals?), but there's no empirical test for it. On the fringes, some people believe only humans (or even only themselves personally) possess it, while other people believe that even inanimate rocks could possess it. So basically we're never going to achieve some consensus about whether an LLM/AI could have it, because we can't even agree on whether dogs and cats have it. We can't even all agree on whether all humans have it.
  - lisper 3y ago
    All this assumes that "it" is a dichotomy: either something has "it" or they do not. But it seems pretty clear to me that "it" must be a continuum even in humans because it is very hard to draw a sharp line between a fertilized egg and an adult where a human comes to have "it". (This is the reason that some people conclude that fertilization must be that line, because there are very few other viable candidates.)
    - mcpackieh 3y ago
      I agree that the line is fuzzy, not sharp. But either way, how do you precisely determine the parameters of the line or gradient experimentally? It's my personal belief that a salmon is conscious, but less so than a dog, and that a preying mantis is conscious, but less so than a salmon. But these aren't beliefs handed to me by science, they're just my guesses informed by some casual knowledge of how complex the brains of these animals are. I assume that anything with a brain can have the experience of consciousness and that this has some relationship to the complexity of the brain, but I can't confidently say much more than that. How does an octopus rank against a dog? I have no clue, their brains are structured very different so it's hard to compare the two. Octopus seem to be very smart but their subjective experience could be very different. That's not something I have insight into. Are orcas more conscious than humans? Their brains are bigger and wrinklier, so I think the answer is definitely maybe! There's a distinct possibility that in the mind of an orca the orca is more aware of itself than humans could possibly ever be, but how can we really know that for sure? I can't experience what an orca experiences, so I can't directly compare the orca mental experience to my own. Studying the behavior of orcas can reveal interesting facts, such as their use of language and complex social relationships, but as evidence of subjective consciousness these are indirect at best. Maybe some day we'll have scientific instruments that will allow one being to 'psychically link' with others and experience the other's subjective sense of being. But until we've got something like that, I think the matter is effectively closed to empirical inquiry.
naveen99 3y ago
Thought vector models ?
jhbadger 3y ago
I think we'll need more than that too, but in humans and other animals it isn't the building blocks that are self-aware -- neurons aren't any more aware than any other cell -- self-awareness is a property of a set of connected neurons, not in the neurons themselves.
esafak 3y ago
We need physical-world model building beyond mere token prediction.

danryan 3y ago

Bingo.

strikelaserclaw 3y ago

Although i fundamentally agree with you, i will say that it is impossible to know which "tricks" will lead to a leap in cognition, even looking evolutionarily, the difference in genome between a monkey and human is very small percentage wise, yet the effects of that small difference are stark.

unshavedyak 3y ago
I'm always on the position that at some point (i make no prediction of when) "AI" will still be labeled as not-intelligent, but to external observers it won't be possible to know. Ie we will be able to describe how a function of `add(1,2)` is most clearly definitely not intelligent. We will understand the internal makeup of these neurons better, the whole system will be deterministic and understood - just a very large `add(1,2)` .. so it's definitely not intelligent, of course. Yet it will still produce output that is fully and truly turing-proof. If you let it run, it retains, stores, "behaves", etc. It's clearly just `add(1,2)` in a loop after all.. we will understand it so well that it's obviously just a trick of our mind that it seems real. The conclusion in that space will not be one of the machine .. but of us. If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us? The existential crisis will be neat.
- c_crank 3y ago
  >If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us? Less than people think. We know that being a biological system differentiates us. If intelligent systems tend to converge where language is concerned, so be it. It doesn't mean the AI warrants any special consideration. It's still a tool.
  - mercurialsolo 3y ago
    While life is typically associated with biological systems, intelligence doesn't necessarily need proof of life. A hive, swarm or network can be intelligent as long as it shows self-awareness and has agency.
    - c_crank 3y ago
      I'm not disputing that. I just think that intelligence or 'sapience' is not something that requires reverence on our part.
      - TeMPOraL 3y ago
        What is then, if anything? Definitely not "being a biological system" - that seems much more boring than intelligence, and the line between "biological" and "non-biological" is somewhat arbitrary and increasingly blurry anyway.
        c_crank 3y ago
        It is a quality that makes something good at drawing pictures and writing smart sound replies to prompts. And translating.
        jacobr1 3y ago
        I'm increasingly thinking the game-theoretic emergent properties of social interaction as described in papers like https://desirism.com/morality-from-the-ground-up https://desirism.com/morality-from-the-ground-up provides a good basis for thinking about this. The distinguishing factor then becomes something like, the agent is is participant in the moral-sphere by having "enough" properties to do so. It still leaves the details of that gradient a bit undefined, but at least specifies the basis upon which to make a judgment.

ImHereToVote 3y ago

Why not use less emotionally charged words? Goal to action mapping for instance. How far away are we from a general goal to action mapper that can outcompete a human being on a given goal.

indigo945 3y ago

I am an absolute layman when it comes to AI topics, but I have been wondering about this lately. A lot of the hopes in regard to AGI in LLMs seem to focus on just making the models bigger, or feeding them even more information. On the other hand, some researchers in the AGI space seem to move in a completely different direction, trying to use neural networks to simulate different parts of a brain, moving away from language altogether. But what I'm wondering is this: might it be a more productive avenue to consider an LLM not as an entire mind, but as a single part of an hypothetical artifical psyche, which excels at some of the overarching tasks of consciousness (such as acting within a cultural context and epistemological framework), but can delegate other aspects to more fitting subsystems? Could it be combined with other networks optimized for different tasks? Potentially, this could just be done by having inputs from other parts of this artificial psyche fed in as prompts, translated by an encoder/decoder of the likes of CLIP, i.e. a decoder that can translate ideas between different liminal spaces. The work that I keep remembering in this context is Jaynes' Origin of Consciousness in the Breakdown of the Bicameral Mind [1], which I stumbled upon here on HN [2] [3]. Jaynes believes that consciousness is not an innate ability, but a cultural achievement - in order to become conscious, one first has to learn (through others) to conceptualize consciousness, and to think of oneself as a thinking being. He insists that consciousness is not a necessity for human agency, and that humanity has been building civilizations and using technologies long before humans discovered (or achieved) consciousness. Jaynes puts forward the idea of an ancient, non-conscious human mind based on, upon other things, "verbal hallucinations" - thoughts and memories that involuntarily enter as associations, but whose origins are interpreted by the mind to not be within itself, but somewhere outside of it, such as in a divine monologue; thought, to the pre-conscious mind, is the incessant blabbering of the Gods. It seems that LLMs would excel in the role of this "camera" of the mind - as the source of divine monologue and divine instructions, that can then be compared with memories, interpreted and executed by more specialized subsystems. Might it be feasible to give up the plan to make AGI self-aware altogether and instead build an AGI powered by a GPT-4 ascended to divinity, just as ignorant of its own existence as ever? [1]: https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in_the_Breakdown_of_the_Bicameral_Mind https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in... [2]: https://news.ycombinator.com/item?id=27917316 https://news.ycombinator.com/item?id=27917316 [3]: https://news.ycombinator.com/item?id=27923444 https://news.ycombinator.com/item?id=27923444

empath-nirvana 3y ago

what is the 'real deal', exactly? I feel like people have this vague idea in their head of what everyone is trying to build, but don't really have it defined in a concrete way.

mcpackieh 3y ago
The fundamental issue is that all the talk of "self-awareness", "cognition", "souls", whatever you want to call it, is all beating around the bush to describe what we presumably each subjectively experience in our own minds but which cannot be empirically tested for in others. We generally assume that other people experience the same, but cannot empirically prove it and some people speculate that others might lack it (P-zombies, "NPCs", etc). And then when it comes to animals, a huge portion of the population refuse to believe animals can possess it. I think the majority belief is that some animals definitely have it (dolphins) and some animals probably do not (jellyfish? corals?), but there's no empirical test for it. On the fringes, some people believe only humans (or even only themselves personally) possess it, while other people believe that even inanimate rocks could possess it. So basically we're never going to achieve some consensus about whether an LLM/AI could have it, because we can't even agree on whether dogs and cats have it. We can't even all agree on whether all humans have it.
- lisper 3y ago
  All this assumes that "it" is a dichotomy: either something has "it" or they do not. But it seems pretty clear to me that "it" must be a continuum even in humans because it is very hard to draw a sharp line between a fertilized egg and an adult where a human comes to have "it". (This is the reason that some people conclude that fertilization must be that line, because there are very few other viable candidates.)
  - mcpackieh 3y ago
    I agree that the line is fuzzy, not sharp. But either way, how do you precisely determine the parameters of the line or gradient experimentally? It's my personal belief that a salmon is conscious, but less so than a dog, and that a preying mantis is conscious, but less so than a salmon. But these aren't beliefs handed to me by science, they're just my guesses informed by some casual knowledge of how complex the brains of these animals are. I assume that anything with a brain can have the experience of consciousness and that this has some relationship to the complexity of the brain, but I can't confidently say much more than that. How does an octopus rank against a dog? I have no clue, their brains are structured very different so it's hard to compare the two. Octopus seem to be very smart but their subjective experience could be very different. That's not something I have insight into. Are orcas more conscious than humans? Their brains are bigger and wrinklier, so I think the answer is definitely maybe! There's a distinct possibility that in the mind of an orca the orca is more aware of itself than humans could possibly ever be, but how can we really know that for sure? I can't experience what an orca experiences, so I can't directly compare the orca mental experience to my own. Studying the behavior of orcas can reveal interesting facts, such as their use of language and complex social relationships, but as evidence of subjective consciousness these are indirect at best. Maybe some day we'll have scientific instruments that will allow one being to 'psychically link' with others and experience the other's subjective sense of being. But until we've got something like that, I think the matter is effectively closed to empirical inquiry.

naveen99 3y ago

Thought vector models ?

jhbadger 3y ago

I think we'll need more than that too, but in humans and other animals it isn't the building blocks that are self-aware -- neurons aren't any more aware than any other cell -- self-awareness is a property of a set of connected neurons, not in the neurons themselves.

esafak 3y ago

We need physical-world model building beyond mere token prediction.