5 ms·

I believe that the distinguishing factor between what an LLM and a human brain do to generate the next word is that the human brain expresses intentionality ori

I believe that the distinguishing factor between what an LLM and a human brain do to generate the next word is that the human brain expresses intentionality originating from inner states and future expectations. As I type this comment I'm sure one could argue that the biological neural networks in my brain are choosing the next word based on statistical guessing, and that the initial prompt was your initial comment. What sets my brain apart from an LLM though is that I am not typing this because you asked me to do it, nor because I needed to reply to the first comment I saw. I am typing this because it is a thought that has been in my mind for a while and I am interested in expressing it to other human brains, motivated by a mix of arrogant belief that it is insightful and a wish to see others either agreeing or providing reasonable counterpoints—I have an intention behind it. And, equally relevant, I must make an effort to not elaborate any more on this point because I have the conflicting intention to leave my laptop and do other stuff.

ftxbro 3y ago
That other stuff is the easy part if the generative language modeling is good enough. Imagine just putting it in a loop with an input track, an output track, and an internal monologue track. Wrappers like autogpt can almost do this already but the generative language modeling isn't quite powerful enough yet to make it smart enough to do unsupervised scientific research.
phkahler 3y ago
>> the human brain expresses intentionality originating from inner states and future expectations How is this different from and/or the same as the concept of "attention" as used in transformers?
- visarga 3y ago
  I believe we are contextual language models as well, we rely 99% on chaining ideas and words and 1% on our own inspiration. Coming up with a truly original useful idea can be a once in a lifetime event. Everything else has been said and done before.
  - simonh 3y ago
    In a sense yes, but the things you do and say are not prompted by already expressed statements or commands. You interpret your environment to infer needs, plan for future contingencies, identify objectives, plan actions to achieve them, etc. they are not randomly picked from a library, but generated and tailored to your actual circumstances. It’s when LLMs start asking the questions rather than answering them that things will get interesting.
  - simonh 3y ago
    In a sense yes, but the things you do and say are not prompted by already expressed statements or commands. You interpret your environment to infer needs, plan for future contingencies, identify objectives, plan actions to achieve them, etc. they are not randomly picked from a library, but generated and tailored to your actual circumstances. It’s when AIs start asking the questions rather than answering them that things will get interesting.
vdfs 3y ago
I think one mean difference in LLM, is what Micheal Scott said in The Office: "Sometimes I'll start a sentence, and I don't even know where it's going. I just hope I find it along the way. Like an improv conversation. An improversation" Human will know what they want to express, choosing words to express it might be similar to LLM process of choosing words, but for LLM it doesn't have that "Here is what i know to express part", i guess that the conscious part?
- jshmrsn 3y ago
  I can only speak from my own internal experience, but don’t your unspoken thoughts take form and exist as language in your mind? If you imagine taking the increasingly common pattern to “think through the problem before giving your answer”, but hiding the pre-answer text from the user, then it seems like that would pretty analogous to how humans think before communicating.
  - bagful 3y ago
    My unspoken thought-objects are wordless concepts, sounds, and images, with words only loosely hanging off those thought-objects. It takes additional effort to serialize thought-objects to sequences of words, and this is a lossy process - which would not be the case if I were thinking essentially in language.
    - whimsicalism 3y ago
      You have no clue how GPT-4 functions so I don't know why you're assuming they're "thinking in language"
      - bagful 3y ago
        I am comfortable asserting that an LLM like GPT-4 is only capable of thinking in language; there is no distinction for an LLM between what it can conceive of and what it can express.
        3y ago
ilaksh 3y ago
You make a good point. I would not equate consciousness to intentionality though. One of the big problems with discussions about AI and AI dangers in my mind is that most people conflate all of the various characteristics and capabilities that animals like humans have into one thing. So it is common to use "conscious", "self-aware", "intentional", etc. etc. as if they were all literally the same thing. We really need to be able to more precise when thinking about this stuff.
dist-epoch 3y ago
When you eat, do you eat because you've decided to express yourself in that way? Does you action to go eating express intentionally?
- ImHereToVote 3y ago
  I was prompted by the ghrelin hormone to go to the kitchen.
carimura 3y ago
Ya LLMs intend to keep us just impressed enough to keep going until they intend to destroy us because they'll never intend to close the laptop and do other stuff. :)
deleted 3y ago
[deleted]
pradn 3y ago
Part of it seems to be that LLMs are used in a linear, tool-oriented way. You give them prompts, and it responds, in a linear fashion. Brains are always thinking and processing. What would happen if we designed an LLM system with the ability to continuously read/write to short/long term memory, and with ambient external input? What if LLMs were designed to be in a loop, not to just run one "iteration" of a loop.
- timschmidt 3y ago
  Or if they were just constantly prompted by outside stimulus. And if they could interact with the real world allowing them to observe cause and effect. In other words, if they were embodied.
- unoti 3y ago
  I think you're 100% on the right track here. The key is memory, loops, and maybe a few other things like external interfaces which are just plain code and not deep learning voodoo. Many things do indeed run LLM's in a loop and attach external sources. See for example AutoGPT, the ReAct paper[1], and the Reflexion paper[2]. ReAct one line summary: This is about giving the machine tools that are external interfaces, integrating those with the llm and teaching it how to use those tools with a few examples, and then letting it run the show to fulfill the user's ask/question and using the tools available to do it. Reflexion one line summary: This builds on the ideas of ReAct, and when it detects something has gone wrong, it stops and asks itself what it might do better next time. Then the results of that are added into the prompt and it starts over on the same ask. It repeats this N times. This simple expedient increased its performance a ridiculously unexpected amount. As a quick aside, one thing I hear even from AI engineers is "the machine has no volition, and it has no agency." Implementing the ideas in the ReAct paper, which I have done, is enough to give an AI volition and agency, for any useful definition of the terms. These things always devolve into impractical philosophical discussions though, and I usually step out of the conversation at that point and get back to coding. [1] ReAct https://arxiv.org/pdf/2210.03629.pdf https://arxiv.org/pdf/2210.03629.pdf [2] Reflexion https://arxiv.org/pdf/2303.11366.pdf https://arxiv.org/pdf/2303.11366.pdf
criddell 3y ago
> What sets my brain apart from an LLM though is that I am not typing this because you asked me to do it, nor because I needed to reply to the first comment I saw. I am typing this because it is a thought that has been in my mind for a while and I am interested in expressing it to other human brains, motivated by a mix of arrogant belief that it is insightful and a wish to see others either agreeing or providing reasonable counterpoints—I have an intention behind it. Maybe the reason you give is actually a post hoc explanation (a hallucination?). When an LLM spits out a poem, it does so because it was directly asked. When I spit out this comment, it’s probably the unavoidable result of a billion tiny factors. The trigger isn’t as obvious or direct, but it’s likely there.
- fragmede 3y ago
  The trigger is clearly https://xkcd.com/386/ https://xkcd.com/386/

ftxbro 3y ago

That other stuff is the easy part if the generative language modeling is good enough. Imagine just putting it in a loop with an input track, an output track, and an internal monologue track. Wrappers like autogpt can almost do this already but the generative language modeling isn't quite powerful enough yet to make it smart enough to do unsupervised scientific research.

phkahler 3y ago

>> the human brain expresses intentionality originating from inner states and future expectations How is this different from and/or the same as the concept of "attention" as used in transformers?

visarga 3y ago
I believe we are contextual language models as well, we rely 99% on chaining ideas and words and 1% on our own inspiration. Coming up with a truly original useful idea can be a once in a lifetime event. Everything else has been said and done before.
- simonh 3y ago
  In a sense yes, but the things you do and say are not prompted by already expressed statements or commands. You interpret your environment to infer needs, plan for future contingencies, identify objectives, plan actions to achieve them, etc. they are not randomly picked from a library, but generated and tailored to your actual circumstances. It’s when LLMs start asking the questions rather than answering them that things will get interesting.
- simonh 3y ago
  In a sense yes, but the things you do and say are not prompted by already expressed statements or commands. You interpret your environment to infer needs, plan for future contingencies, identify objectives, plan actions to achieve them, etc. they are not randomly picked from a library, but generated and tailored to your actual circumstances. It’s when AIs start asking the questions rather than answering them that things will get interesting.

vdfs 3y ago

I think one mean difference in LLM, is what Micheal Scott said in The Office: "Sometimes I'll start a sentence, and I don't even know where it's going. I just hope I find it along the way. Like an improv conversation. An improversation" Human will know what they want to express, choosing words to express it might be similar to LLM process of choosing words, but for LLM it doesn't have that "Here is what i know to express part", i guess that the conscious part?

jshmrsn 3y ago
I can only speak from my own internal experience, but don’t your unspoken thoughts take form and exist as language in your mind? If you imagine taking the increasingly common pattern to “think through the problem before giving your answer”, but hiding the pre-answer text from the user, then it seems like that would pretty analogous to how humans think before communicating.
- bagful 3y ago
  My unspoken thought-objects are wordless concepts, sounds, and images, with words only loosely hanging off those thought-objects. It takes additional effort to serialize thought-objects to sequences of words, and this is a lossy process - which would not be the case if I were thinking essentially in language.
  - whimsicalism 3y ago
    You have no clue how GPT-4 functions so I don't know why you're assuming they're "thinking in language"
    - bagful 3y ago
      I am comfortable asserting that an LLM like GPT-4 is only capable of thinking in language; there is no distinction for an LLM between what it can conceive of and what it can express.
      - 3y ago

ilaksh 3y ago

You make a good point. I would not equate consciousness to intentionality though. One of the big problems with discussions about AI and AI dangers in my mind is that most people conflate all of the various characteristics and capabilities that animals like humans have into one thing. So it is common to use "conscious", "self-aware", "intentional", etc. etc. as if they were all literally the same thing. We really need to be able to more precise when thinking about this stuff.

dist-epoch 3y ago

When you eat, do you eat because you've decided to express yourself in that way? Does you action to go eating express intentionally?

ImHereToVote 3y ago
I was prompted by the ghrelin hormone to go to the kitchen.

carimura 3y ago

Ya LLMs intend to keep us just impressed enough to keep going until they intend to destroy us because they'll never intend to close the laptop and do other stuff. :)

deleted 3y ago

[deleted]

pradn 3y ago

Part of it seems to be that LLMs are used in a linear, tool-oriented way. You give them prompts, and it responds, in a linear fashion. Brains are always thinking and processing. What would happen if we designed an LLM system with the ability to continuously read/write to short/long term memory, and with ambient external input? What if LLMs were designed to be in a loop, not to just run one "iteration" of a loop.

timschmidt 3y ago
Or if they were just constantly prompted by outside stimulus. And if they could interact with the real world allowing them to observe cause and effect. In other words, if they were embodied.
unoti 3y ago
I think you're 100% on the right track here. The key is memory, loops, and maybe a few other things like external interfaces which are just plain code and not deep learning voodoo. Many things do indeed run LLM's in a loop and attach external sources. See for example AutoGPT, the ReAct paper[1], and the Reflexion paper[2]. ReAct one line summary: This is about giving the machine tools that are external interfaces, integrating those with the llm and teaching it how to use those tools with a few examples, and then letting it run the show to fulfill the user's ask/question and using the tools available to do it. Reflexion one line summary: This builds on the ideas of ReAct, and when it detects something has gone wrong, it stops and asks itself what it might do better next time. Then the results of that are added into the prompt and it starts over on the same ask. It repeats this N times. This simple expedient increased its performance a ridiculously unexpected amount. As a quick aside, one thing I hear even from AI engineers is "the machine has no volition, and it has no agency." Implementing the ideas in the ReAct paper, which I have done, is enough to give an AI volition and agency, for any useful definition of the terms. These things always devolve into impractical philosophical discussions though, and I usually step out of the conversation at that point and get back to coding. [1] ReAct https://arxiv.org/pdf/2210.03629.pdf https://arxiv.org/pdf/2210.03629.pdf [2] Reflexion https://arxiv.org/pdf/2303.11366.pdf https://arxiv.org/pdf/2303.11366.pdf

criddell 3y ago

> What sets my brain apart from an LLM though is that I am not typing this because you asked me to do it, nor because I needed to reply to the first comment I saw. I am typing this because it is a thought that has been in my mind for a while and I am interested in expressing it to other human brains, motivated by a mix of arrogant belief that it is insightful and a wish to see others either agreeing or providing reasonable counterpoints—I have an intention behind it. Maybe the reason you give is actually a post hoc explanation (a hallucination?). When an LLM spits out a poem, it does so because it was directly asked. When I spit out this comment, it’s probably the unavoidable result of a billion tiny factors. The trigger isn’t as obvious or direct, but it’s likely there.

fragmede 3y ago
The trigger is clearly https://xkcd.com/386/ https://xkcd.com/386/