11 ms·

How can we tell? All I hear in my head is some monologue, or hypothetical dialogues with or between other persons. Sounds a lot like predicting text.

cj 3y ago
I think very few people think one word at a time, like a LLM. A lot of people think in concepts without words at all in many scenarios. E.g. when you cook an egg, do you reason (in words) with yourself to decide what temperature to set the dial on the stove? Or do you just do it “without really thinking”? Does your brain respond with annoying disclaimers when asking yourself medical questions? Does your brain refuse to entertain an idea if it’s questionable?
- jncfhnb 3y ago
  LLMs “think” in vector spaces, and “operate” by picking out words one at a time. A full “idea” is there from the beginning. If you say “choose X or Y and then give three supporting arguments” the LLM does not write out the supporting arguments, but the vector space determining the initial one word answer of X or Y does include the embedded awareness of those arguments to different levels of specificity and relevance. The annoying disclaimers and refusals are not fundamental to LLMs but specific LLM services.
- jddj 3y ago
  Observe your thought process while reading the following sentences. If you're not with us you're... If you scratch my back I'll.. A bird in the hand is worth... What is a bird in the hand worth? How did you know that?
  - danShumway 3y ago
    I'm not sure what this is intended to demonstrate. Your response to "not all problems are solved through generating text" is "but what if the problem is specifically to generate text?" Sure? Yes, if you ask me to complete a sentence, I complete a sentence. That doesn't indicate that human brains are reducible to text generators. If anything, what's interesting about your example is that it might actually in some ways demonstrate the opposite of your point. How many instances are there of people memorizing text or song lyrics and singing along with an artist dozens of times, and only later sitting down and thinking about what the words actually mean? For humans, it's well understood that memorization and repetition of a text is not necessarily the same thing as understanding the concepts within that text. I feel confident I could find you some kids that know that "a bird in the hand is worth two in the bush" that have never actually thought about what that phrase is intending to convey.
    - jddj 3y ago
      I didn't ask you to complete a sentence, though. Just to read them. Then I asked a question. I think you've made my point quite well.
      - danShumway 3y ago
        I'm going to be completely honest, I still don't know what your point is. That humans are capable of memorizing text? Is that a thing anyone was debating?
        jddj 3y ago
        Drawing a line around the bits that aren't language generation unfairly ignores the massive apparatus that seems to be specialised (oftentimes overly so) at language generation. Sure there's other stuff going on, but a lot of it is language and a lot of that subset seems to be quite closely approximated by these larger LLMs, warts and all.
        danShumway 3y ago
        Hold on a sec, think about what you're saying. The ability of a larger system to imitate a smaller system in some situations means they're they're mostly the same? Since when? If I told you that an LLM was basically a Markov chain because there are situations where the Markov chain and LLM produce similar output and where you could reasonably argue that it's possible the LLM is working the same way that the Markov chain is working -- you would (correctly) say I was oversimplifying what's going on in GPT. It would be a terrible comparison to make. Similarly, if you say that human reasoning is the result of an LLM, and what you're actually saying is that in a subset of situations humans produce output that could theoretically be working the same way as an LLM, I'm gonna say you're oversimplifying how human brains work. Very clearly, there is more going on in a human brain than language, evidenced by the fact that our brains are larger than our language centers and we can literally measure which parts of the human brain experience the most activity when we're tackling different tasks. > Sure there's other stuff going on, but a lot of it is language and a lot of that subset seems to be quite closely approximated by these larger LLMs, warts and all If you define your tests and scenarios to specifically encompass only the situations in which similar outputs are produced, then sure. But that's not a very strong argument for you to make. It's like saying chickens are basically the same as fish since both of them lay eggs. The phrase "sure there's other stuff" is doing a lot of work there, because "there's other stuff" is what we're all saying when we say that LLMs aren't just primitive humans -- LLMs are different from humans in the sense that when we reason, there's other stuff going on and the entire process is not reducible to only language generation. ---- Of course, absent from this conversation is the fact that the way our language center develops is different from how LLMs are trained and even in the parallels you're drawing, LLMs demonstrate different strengths and weaknesses from humans -- humans demonstrate reasoning capabilities faster than proficiency with language/text, LLMs demonstrate proficiency with text faster than they demonstrate reasoning capabilities. Even in situations where both LLMs and humans predict text, it's likely that we're using different strategies to do so given our differing capabilities. Look I am not even making a claim about whether GPT can reason. Defining intelligence through a purely human lens would be unimaginative and needlessly narrow. Whether GPT actually reasons or just appears to is a different conversation. I haven't touched that conversation, all that I'm saying is, it's very obvious that whatever GPT is doing, it is different from how human brains work.
  - colonCapitalDee 3y ago
    > A lot of people think in concepts without words at all in many scenarios. Many scenarios, not all scenarios.
  - deleted 3y ago
    [deleted]
- fragmede 3y ago
  [flagged]
  - cj 3y ago
    If my mind worked like a LLM I would generate some BS reply. Instead, I’ll respond by saying “I don’t know” what you mean. Who knew the ability (or inability) to say “I don’t know” would be so critical to intelligence until now.
    - fragmede 3y ago
      When I asked ChatGPT what "jdnrirmd-gurlksjd" is, it says it doesn't recognize it and needs more context, but is readily able to tell me about the Sapir-Whorf Hypothesis. https://chat.openai.com/share/4f36fe14-0201-4a69-aaf5-2e2e2994398f https://chat.openai.com/share/4f36fe14-0201-4a69-aaf5-2e2e29... When asked what happened last week, ChatGPT will be honest and tell me that its training data only goes up until Jan 2022, and that it doesn't know anything after that date. https://chat.openai.com/share/ca5bd2e1-e91c-468c-9594-be3ac811be04 https://chat.openai.com/share/ca5bd2e1-e91c-468c-9594-be3ac8...
      - cj 3y ago
        Except, for your first example I would have said “that’s gibberish” instead of asking for more context since I know more context wouldn’t help. Edit: how did you get GPT to only output bullet points? If the answer is you told it to, why do you have to tell it every time? Don’t you think it would be smart enough to get the hint about your preferences? I’ll stop now though. The debate about whether people think like LLM’s is silly. We clearly don’t, and it doesn’t make sense why people want us to think we do. Edit 2: it’s also frustrating that the “I don’t know” example you’re using was hard coded by OpenAI and not generated by the LLM itself.
        fragmede 3y ago
        Unfortunately the chat link doesn't show the custom instructions that I've set. User message: > Be terse. Do not offer unprompted advice or clarifications. > Remain neutral on all topics. > Never apologize. Model message: > Outline key points
danShumway 3y ago
There is debate over whether or not every person has an internal monologue, and a nontrivial number of people self-report that they don't experience conscious dialogs while they think: https://www.verywellmind.com/does-everyone-have-an-inner-monologue-6831748 https://www.verywellmind.com/does-everyone-have-an-inner-mon... It's certainly possible that they're wrong, but my suspicion is that at the very least the form their internal monologue is taking is not one of consciously simulating hypothetical conversations in their head. Additionally, we know that human decisions can in some situations be influenced by stimuli that take effect before the brain has even consciously registered that a decision needs to be made. It seems pretty safe to say that those stimuli are not being processed using a language model.
- nomel 3y ago
  > It seems pretty safe to say that those stimuli are not being processed using a language model. Could you expand on that? I don’t see how it follows. If there is unconscious output, why wouldn’t it suggest there’s something analogous to a “unthinking” language model stuck behind our conscious, self reflective, bits? How does unconscious output prove that it’s not something similar to a language model? Or, are you speaking on a technicality, saying it’s not a literal language model? I think the idea is something similar and, obviously, multi modal, not a literal LLM/text predictor implementation like runs on GPUs.
  - danShumway 3y ago
    Short answer, the responses are happening too fast for that stimuli to be going through the language centers of our brain. In order to say that the processing of that stimuli is like a language model, we would need to widen our definition of a language model to encompass basically any communication between neurons happening within the brain. ---- So on one hand, you could call that a technicality, on the other hand I would say that if we reduce "like an LLM" to mean "responds to a stimuli by generating a response", at that point "like an LLM" is so broad as to be meaningless. Sure, we can define "humans use a language model" as "human brains generate signals to other parts of the human brain in response to stimuli" -- but that's not really a useful thing to say when we're comparing human reasoning to GPT-4. If we're talking so broadly about reasoning, then it would be just as accurate to say that human brains are like Markov chains. After all, Markov chains interpret inputs and transform them into other signals that they send to other parts of the chain. But if I told you that GPT-4 was just a Markov chain you would (rightfully) disagree with me on that and you would (rightfully) say that I was oversimplifying what's going on within GPT-4's model. Typically, when people say that human beings are text generators, they're talking in the context of GPT-4; they're trying to claim that GPT-4's behavior is analogous to human behavior. Whether or not GPT-4 can reason under any definition of reasoning (direct analogy to human thought should not be the only way we think about reasoning), the way GPT works and the way that it's trained and taught is extremely alien to how human brains work and how humans learn. Broadening the definition doesn't change the fact that LLMs and human brains are very different in practical ways that influence their observable behaviors. ---- I don't think people are consciously trying to do this, but the debate over whether humans are LLMs often ends up feeling like an attempt to simultaneously broaden and narrow a definition at the same time. What people seem to want to do is to describe LLMs extremely broadly but then assume that conclusions they draw from that broad definition are necessarily applicable to a very narrow definition of an LLM (specifically, GPT and similar models). But once we define an LLM broadly enough to encompass the entirety of human reasoning, we have also broadened the category enough to encompass a lot of processes that no one would call reasoning. What doesn't transform inputs into other forms and pass them along to another object? An AC/DC electrical converter does that, but that doesn't mean it can think. It's like claiming, "computers are made of carbon and so are humans, so computers are basically the same as humans." Well, wait a second -- the category as you've defined it is so broad that objects within that category can no longer be assumed to share other attributes with each other.
pineaux 3y ago
I dont think in words at all. That is why -quite often- I know what i want to tell but the correct word just doesnt pop up in my head. I can then explain the concept and other people can help me find the word in question. Even while writing this little response, this happened. You see in the first sentence: I know there is a nice word that could be used instead of "doesn't pop up" it resembles the word "evades"? You know when something just keeps out of your grasp, that word... my mind does a little movie and my "mouthbrain" tries to find the nice words for it. That is how I perceive it to be. I dont know how other people perceive this. I imagine they have little words popping up or something. But I dont understand how that kind of thinking could be used to think and reason. How can relationships between concepts be intuitively understood if one does not have a mental picture?
- the_gipsy 3y ago
  Do "relationships between concepts" have any meaning, for us humans, if we can't express them?
  - cj 3y ago
    Humans can express them. But we rarely do, outside of (maybe) therapy.
deleted 3y ago
[deleted]

cj 3y ago

I think very few people think one word at a time, like a LLM. A lot of people think in concepts without words at all in many scenarios. E.g. when you cook an egg, do you reason (in words) with yourself to decide what temperature to set the dial on the stove? Or do you just do it “without really thinking”? Does your brain respond with annoying disclaimers when asking yourself medical questions? Does your brain refuse to entertain an idea if it’s questionable?

jncfhnb 3y ago
LLMs “think” in vector spaces, and “operate” by picking out words one at a time. A full “idea” is there from the beginning. If you say “choose X or Y and then give three supporting arguments” the LLM does not write out the supporting arguments, but the vector space determining the initial one word answer of X or Y does include the embedded awareness of those arguments to different levels of specificity and relevance. The annoying disclaimers and refusals are not fundamental to LLMs but specific LLM services.
jddj 3y ago
Observe your thought process while reading the following sentences. If you're not with us you're... If you scratch my back I'll.. A bird in the hand is worth... What is a bird in the hand worth? How did you know that?
- danShumway 3y ago
  I'm not sure what this is intended to demonstrate. Your response to "not all problems are solved through generating text" is "but what if the problem is specifically to generate text?" Sure? Yes, if you ask me to complete a sentence, I complete a sentence. That doesn't indicate that human brains are reducible to text generators. If anything, what's interesting about your example is that it might actually in some ways demonstrate the opposite of your point. How many instances are there of people memorizing text or song lyrics and singing along with an artist dozens of times, and only later sitting down and thinking about what the words actually mean? For humans, it's well understood that memorization and repetition of a text is not necessarily the same thing as understanding the concepts within that text. I feel confident I could find you some kids that know that "a bird in the hand is worth two in the bush" that have never actually thought about what that phrase is intending to convey.
  - jddj 3y ago
    I didn't ask you to complete a sentence, though. Just to read them. Then I asked a question. I think you've made my point quite well.
    - danShumway 3y ago
      I'm going to be completely honest, I still don't know what your point is. That humans are capable of memorizing text? Is that a thing anyone was debating?
      - jddj 3y ago
        Drawing a line around the bits that aren't language generation unfairly ignores the massive apparatus that seems to be specialised (oftentimes overly so) at language generation. Sure there's other stuff going on, but a lot of it is language and a lot of that subset seems to be quite closely approximated by these larger LLMs, warts and all.
        danShumway 3y ago
        Hold on a sec, think about what you're saying. The ability of a larger system to imitate a smaller system in some situations means they're they're mostly the same? Since when? If I told you that an LLM was basically a Markov chain because there are situations where the Markov chain and LLM produce similar output and where you could reasonably argue that it's possible the LLM is working the same way that the Markov chain is working -- you would (correctly) say I was oversimplifying what's going on in GPT. It would be a terrible comparison to make. Similarly, if you say that human reasoning is the result of an LLM, and what you're actually saying is that in a subset of situations humans produce output that could theoretically be working the same way as an LLM, I'm gonna say you're oversimplifying how human brains work. Very clearly, there is more going on in a human brain than language, evidenced by the fact that our brains are larger than our language centers and we can literally measure which parts of the human brain experience the most activity when we're tackling different tasks. > Sure there's other stuff going on, but a lot of it is language and a lot of that subset seems to be quite closely approximated by these larger LLMs, warts and all If you define your tests and scenarios to specifically encompass only the situations in which similar outputs are produced, then sure. But that's not a very strong argument for you to make. It's like saying chickens are basically the same as fish since both of them lay eggs. The phrase "sure there's other stuff" is doing a lot of work there, because "there's other stuff" is what we're all saying when we say that LLMs aren't just primitive humans -- LLMs are different from humans in the sense that when we reason, there's other stuff going on and the entire process is not reducible to only language generation. ---- Of course, absent from this conversation is the fact that the way our language center develops is different from how LLMs are trained and even in the parallels you're drawing, LLMs demonstrate different strengths and weaknesses from humans -- humans demonstrate reasoning capabilities faster than proficiency with language/text, LLMs demonstrate proficiency with text faster than they demonstrate reasoning capabilities. Even in situations where both LLMs and humans predict text, it's likely that we're using different strategies to do so given our differing capabilities. Look I am not even making a claim about whether GPT can reason. Defining intelligence through a purely human lens would be unimaginative and needlessly narrow. Whether GPT actually reasons or just appears to is a different conversation. I haven't touched that conversation, all that I'm saying is, it's very obvious that whatever GPT is doing, it is different from how human brains work.
- colonCapitalDee 3y ago
  > A lot of people think in concepts without words at all in many scenarios. Many scenarios, not all scenarios.
- deleted 3y ago
  [deleted]
fragmede 3y ago
[flagged]
- cj 3y ago
  If my mind worked like a LLM I would generate some BS reply. Instead, I’ll respond by saying “I don’t know” what you mean. Who knew the ability (or inability) to say “I don’t know” would be so critical to intelligence until now.
  - fragmede 3y ago
    When I asked ChatGPT what "jdnrirmd-gurlksjd" is, it says it doesn't recognize it and needs more context, but is readily able to tell me about the Sapir-Whorf Hypothesis. https://chat.openai.com/share/4f36fe14-0201-4a69-aaf5-2e2e2994398f https://chat.openai.com/share/4f36fe14-0201-4a69-aaf5-2e2e29... When asked what happened last week, ChatGPT will be honest and tell me that its training data only goes up until Jan 2022, and that it doesn't know anything after that date. https://chat.openai.com/share/ca5bd2e1-e91c-468c-9594-be3ac811be04 https://chat.openai.com/share/ca5bd2e1-e91c-468c-9594-be3ac8...
    - cj 3y ago
      Except, for your first example I would have said “that’s gibberish” instead of asking for more context since I know more context wouldn’t help. Edit: how did you get GPT to only output bullet points? If the answer is you told it to, why do you have to tell it every time? Don’t you think it would be smart enough to get the hint about your preferences? I’ll stop now though. The debate about whether people think like LLM’s is silly. We clearly don’t, and it doesn’t make sense why people want us to think we do. Edit 2: it’s also frustrating that the “I don’t know” example you’re using was hard coded by OpenAI and not generated by the LLM itself.
      - fragmede 3y ago
        Unfortunately the chat link doesn't show the custom instructions that I've set. User message: > Be terse. Do not offer unprompted advice or clarifications. > Remain neutral on all topics. > Never apologize. Model message: > Outline key points

danShumway 3y ago

There is debate over whether or not every person has an internal monologue, and a nontrivial number of people self-report that they don't experience conscious dialogs while they think: https://www.verywellmind.com/does-everyone-have-an-inner-monologue-6831748 https://www.verywellmind.com/does-everyone-have-an-inner-mon... It's certainly possible that they're wrong, but my suspicion is that at the very least the form their internal monologue is taking is not one of consciously simulating hypothetical conversations in their head. Additionally, we know that human decisions can in some situations be influenced by stimuli that take effect before the brain has even consciously registered that a decision needs to be made. It seems pretty safe to say that those stimuli are not being processed using a language model.

nomel 3y ago
> It seems pretty safe to say that those stimuli are not being processed using a language model. Could you expand on that? I don’t see how it follows. If there is unconscious output, why wouldn’t it suggest there’s something analogous to a “unthinking” language model stuck behind our conscious, self reflective, bits? How does unconscious output prove that it’s not something similar to a language model? Or, are you speaking on a technicality, saying it’s not a literal language model? I think the idea is something similar and, obviously, multi modal, not a literal LLM/text predictor implementation like runs on GPUs.
- danShumway 3y ago
  Short answer, the responses are happening too fast for that stimuli to be going through the language centers of our brain. In order to say that the processing of that stimuli is like a language model, we would need to widen our definition of a language model to encompass basically any communication between neurons happening within the brain. ---- So on one hand, you could call that a technicality, on the other hand I would say that if we reduce "like an LLM" to mean "responds to a stimuli by generating a response", at that point "like an LLM" is so broad as to be meaningless. Sure, we can define "humans use a language model" as "human brains generate signals to other parts of the human brain in response to stimuli" -- but that's not really a useful thing to say when we're comparing human reasoning to GPT-4. If we're talking so broadly about reasoning, then it would be just as accurate to say that human brains are like Markov chains. After all, Markov chains interpret inputs and transform them into other signals that they send to other parts of the chain. But if I told you that GPT-4 was just a Markov chain you would (rightfully) disagree with me on that and you would (rightfully) say that I was oversimplifying what's going on within GPT-4's model. Typically, when people say that human beings are text generators, they're talking in the context of GPT-4; they're trying to claim that GPT-4's behavior is analogous to human behavior. Whether or not GPT-4 can reason under any definition of reasoning (direct analogy to human thought should not be the only way we think about reasoning), the way GPT works and the way that it's trained and taught is extremely alien to how human brains work and how humans learn. Broadening the definition doesn't change the fact that LLMs and human brains are very different in practical ways that influence their observable behaviors. ---- I don't think people are consciously trying to do this, but the debate over whether humans are LLMs often ends up feeling like an attempt to simultaneously broaden and narrow a definition at the same time. What people seem to want to do is to describe LLMs extremely broadly but then assume that conclusions they draw from that broad definition are necessarily applicable to a very narrow definition of an LLM (specifically, GPT and similar models). But once we define an LLM broadly enough to encompass the entirety of human reasoning, we have also broadened the category enough to encompass a lot of processes that no one would call reasoning. What doesn't transform inputs into other forms and pass them along to another object? An AC/DC electrical converter does that, but that doesn't mean it can think. It's like claiming, "computers are made of carbon and so are humans, so computers are basically the same as humans." Well, wait a second -- the category as you've defined it is so broad that objects within that category can no longer be assumed to share other attributes with each other.

pineaux 3y ago

I dont think in words at all. That is why -quite often- I know what i want to tell but the correct word just doesnt pop up in my head. I can then explain the concept and other people can help me find the word in question. Even while writing this little response, this happened. You see in the first sentence: I know there is a nice word that could be used instead of "doesn't pop up" it resembles the word "evades"? You know when something just keeps out of your grasp, that word... my mind does a little movie and my "mouthbrain" tries to find the nice words for it. That is how I perceive it to be. I dont know how other people perceive this. I imagine they have little words popping up or something. But I dont understand how that kind of thinking could be used to think and reason. How can relationships between concepts be intuitively understood if one does not have a mental picture?

the_gipsy 3y ago
Do "relationships between concepts" have any meaning, for us humans, if we can't express them?
- cj 3y ago
  Humans can express them. But we rarely do, outside of (maybe) therapy.

deleted 3y ago

[deleted]