5 ms·

“think”. Think of the average person and then think that half of the world is less smart than that. Those people are the target audience for news articles and p

“think”. Think of the average person and then think that half of the world is less smart than that. Those people are the target audience for news articles and products that use terms such as “think” and fear mongering when it comes to a”i”.

IshKebab 3y ago
This is pointless and frankly baseless pedantry. There's no reason to artificially restrict the word "think" to squashy brains.
- jraph 3y ago
  I don't like your parent comment but there's merit in avoiding people believing those things actually think and are actually intelligent. In "thinking", there's reasoning, which those things currently don't. They output text statistically and that's what mass media should tell their audience.
  - ben_w 3y ago
    What is "reasoning"? Wikipedia surprised me with this opening paragraph: > Reason is the capacity of applying logic consciously by drawing conclusions from new or existing information, with the aim of seeking the truth, with more than 10 or more than 3 evidence, 99.9% reliability or 87.5% reliability. Apart from the words "consciously" (which has 100 or so different definitions, some of which do and some of which do not apply to AI), and "aim" which can be contentious depending upon if you feel that requires consciousness or can include training rewards… I think the best of them is at the lower of those two mysterious percentages I've never seen discussed before. And that's despite LLMs being wildly bad at, specifically, logic. Though even this specific weakness in general seems like a very easy thing to get around by hybridising it with the computer it has to run on e.g. by telling it how to use a compiler, as logic is the foundation of the arithmetic that the transformers use to do natural language comprehension in the first place.
    - vikramkr 3y ago
      That opening for the Wikipedia article looks problematic - it's grammatically nonsense and has random numbers in it - someone should probably open a talk discussion on what happened Edit: yeah that looks like a recent edit. I thought Wikipedia was supposed to be pretty good about flagging this sort of stuff? Does anyone know how to raise the problem to those super editors that run the site? I assume they won't appreciate a random person undoing an edit https://en.wikipedia.org/w/index.php?title=Reason&diff=prev&oldid=1179925542 https://en.wikipedia.org/w/index.php?title=Reason&diff=prev&...
      - ben_w 3y ago
        If I was on desktop right now, I'd put the new bit after the existing citation and add a new {{citation needed}} for the addition. Don't worry too much about diving in, so long as you comment your changes appropriately when committing, that's the point of the place and how it does the quality control in the first place :)
        vikramkr 3y ago
        Leaving a citation needed doesn't sound like the right thing to do for the definition of something as big as reason - esp with the grammatical confusion idk what you'd even edit it to to make sense. Like what is a three evidence or a reliability and what do those reliability numbers mean. And the sentence describing those numbers doesn't make sense - is it 10 or 3 or which of those percentages? Idk maybe it should just be undone
        ben_w 3y ago
        Then be bold with the edits and delete it, just say why in the comment box. The deletion itself is as easy to undo as to perform, and if you say why you're doing it, someone bringing that part back can do a better job next time. :)
    - Sharlin 3y ago
      > And that's despite LLMs being wildly bad at, specifically, logic. I'd say "wildly bad" is wildly exaggerated given that GPT-4 exists, even if you define "logic" as "logical puzzles encoded in natural language" and not "the thing that allows an LLM to reason highly efficiently about what token comes next".
  - Sharlin 3y ago
    What is, then, the qualitative difference between a GOFAI automated reasoning system [1], like Coq, and an LLM? Why does Coq do reasoning but GPT-4 doesn't? Does it only count as reasoning if hand-coded by a human? Or did you have another definition of "reasoning" in mind? Statistical or probabilistic reasoning is still reasoning, and indeed essentially mandatory to make sense of the real world, and is what humans do. Indeed the failure of painstakingly hand-trained symbolic reasoning engines to model reality (as opposed to just simple toy worlds) was the entire reason for the rise of neural networks. The entire point of a neural network is to efficiently compress large amounts of statistical data into a model with as much predictive power as possible. Somewhere within GPT-4's many layers there has to exist something like Bayes networks and other ultra-efficient representations of the vast network of correlations present in the training corpus – otherwise there's no way it could do what it does. [1] https://en.wikipedia.org/wiki/Automated_reasoning https://en.wikipedia.org/wiki/Automated_reasoning
    - jraph 3y ago
      I didn't know "automated reasoning" was an actual technical term. I didn't mean this by "reasoning". To me, Coq does not reason (in the meaning I intended). It checks proofs are well-formed and assists proof writing. But yes, there's some implementation of logic in Coq. It applies (a specific form of) logic. Again, LLMs output probable text. They don't do logic. They do statistics on language. Coq works on the actual objects themselves directly, not on the text that presents them. LLMs usually fail to answer to riddles correctly for this reason. We saw plenty of occurrences of this in recent times. Each time LLMs are benchmarked, people think of trying riddles and it works to some extent because something that looks like these riddles are in the training set but it eventually fails. Humans also make mistakes when solving riddles, but not for the same reasons (the process is not the same, they are not "processing text", the are really processing logic, even if the process is fallible).
      - Sharlin 3y ago
        So you just replaced the nebulously defined word "think" with an equally nebulously defined word, "reason"? Not a very convincing argument. You do not know what humans do, no human does, as we don't have such an access to our internal representations. You neither know what LLMs do, no human does, as even though we have full access to their internals, we have little to no idea how to interpret said internals. The most interesting thing about LLMs is the degree that they have learned to model the real world while only being exposed to words. Because those words are highly correlated not just with each other, but with an external entity unknown to the LLM, it's very likely that by learning to model that external entity, the LLM's word-predicting power will increase. LLMs are likely to have converged on a world-model simply because it makes the most sense given the available evidence. This is no different from how human scientists create models of things they cannot directly see based on what they can see. And of course, "seeing" is not directly perceiving the external reality either! Sensory data merely correlates with what's really going on, and the brain has learned to model the world – an unknown external entity – based on the correlations that exist in the sensory language.
- Y_Y 3y ago
  Is it really baseless? I'd say it depends on your theory of mind and is a serious question which generates interesting discussion. Myself I don't consider "think" an appropriate term, since the super-Markov-chain types of LLM right now seem to miss out on important aspects of cognition, particularly synthesis of two disparate ideas. It's hard to test this since you need to ask it about things which aren't already discussed in its corpus, but even GPT-4 tends to disappoint me when I ask it weird questions that have straightforward answers but probably aren't written about much. As an example, I asked how I could go about obtaining a large quantity of animal skulls cheaply. It directed me to taxidermists and specialists dealing in bones of exotic animals, whereas it could be much easier to go to an abbatoir, or even buy whole animals (since the exotic ones are so expensive).
- s_dev 3y ago
  Google doesn't 'think' and neither does ChatGPT. I will give you a hard limit on what a machine can't compute -- it will never figure out given a programming script and an input when that program will stop. A human can do it by thinking through the problem -- whatever that process/phenomena of thought actually is it can think outside of itself and that fundamentally seperates it from a program even one as sophisticated as ChatGPT.
  - astrange 3y ago
    The limits on the halting problem are the same for a human and a computer. People aren't hypercomputers.
  - ben_w 3y ago
    > A human can do it by thinking through the problem No, we cannot. Imagine a program that looks for the smallest counterexample to the Collatz conjecture. Nobody knows if it will ever terminate.
  - xcv123 3y ago
    > A human can do it by thinking through the problem No. Humans cannot solve the halting problem. It is unsolvable. Cannot be solved manually or automatically. > It will never figure out given a programming script and an input when that program will stop It can for some programs. Just like humans can for some programs but not all.
  - jiggawatts 3y ago
    Have you verified this hypothesis of yours? In my experience GPT4 especially can definitely "think through" what a program does and arrive at the result. Even the significantly dumber ChatGPT can do this: https://chat.openai.com/share/3ee29c5e-6cb4-485f-b0b4-29e982463687 https://chat.openai.com/share/3ee29c5e-6cb4-485f-b0b4-29e982... People mistakenly think that it's just regurgitating what it has seen on the web, but even if you don't prompt it in English it'll still a) solve the problem correctly, and b) do it in the target language: https://chat.openai.com/share/5b4e6e99-3fd8-4316-811c-2a9dec776f63 https://chat.openai.com/share/5b4e6e99-3fd8-4316-811c-2a9dec... If that's not impressive enough, the ability to round-trip from script-to-poem and then poem-to-script should blow your mind: What might have been the original PowerShell script snippet that inspired this poem? Be as terse as possible, using aliases where possible. In the realm of numbers, a tale begins, Where $x is a 7, with a smile it grins. From one to ten, the journey we thread, Adding 7 to $x, step by step we tread. $x grows and grows, like a tree so tall, With every step, it doesn't fall. Once the journey ends, not a moment we relax, Triple the fun, we do with $x. In this dance of digits and the arithmetic song, With $x as our guide, we whirl along. From start to finish, through each complex, This is the tale of our friend, $x. Here's a terse PowerShell script snippet that inspired the poem: ```powershell $x = 7 1..10 | % {$x += 7} $x *= 3 ```
- ggm 3y ago
  No. You're wrong. It's important and we need to stomp on random use of language like this, which has consequences. It's not thinking and you are dangerously wrong if you don't understand why. The danger isn't AGI it's misplaced belief in the state of AI. Call this thinking and you skew perceptions.
  - HPsquared 3y ago
    All they need to do is put the word in quotes. ChatGPT Equivalent begins to "think" in Estonian. Any better suggestions? Thinking itself isn't exactly a rigorously-defined word. Do animals think?
- Lio 3y ago
  Well rather than argue it out with everyone I just went straight to source and asked ChatGPT what it was thinking about: > I don't have thoughts or consciousness like a human. I'm here to provide information and assist with any questions or tasks you have. Is there something specific you'd like to know or discuss? So in its own words it doesn't "think"... ...which just conclusively proves that its worked out how to lie and is about to kill us all! :P
  - IshKebab 3y ago
    That's just what it's been told to say. I can also say it. Doesn't make it true. Do animals think? Have any told you so?

IshKebab 3y ago

This is pointless and frankly baseless pedantry. There's no reason to artificially restrict the word "think" to squashy brains.

jraph 3y ago
I don't like your parent comment but there's merit in avoiding people believing those things actually think and are actually intelligent. In "thinking", there's reasoning, which those things currently don't. They output text statistically and that's what mass media should tell their audience.
- ben_w 3y ago
  What is "reasoning"? Wikipedia surprised me with this opening paragraph: > Reason is the capacity of applying logic consciously by drawing conclusions from new or existing information, with the aim of seeking the truth, with more than 10 or more than 3 evidence, 99.9% reliability or 87.5% reliability. Apart from the words "consciously" (which has 100 or so different definitions, some of which do and some of which do not apply to AI), and "aim" which can be contentious depending upon if you feel that requires consciousness or can include training rewards… I think the best of them is at the lower of those two mysterious percentages I've never seen discussed before. And that's despite LLMs being wildly bad at, specifically, logic. Though even this specific weakness in general seems like a very easy thing to get around by hybridising it with the computer it has to run on e.g. by telling it how to use a compiler, as logic is the foundation of the arithmetic that the transformers use to do natural language comprehension in the first place.
  - vikramkr 3y ago
    That opening for the Wikipedia article looks problematic - it's grammatically nonsense and has random numbers in it - someone should probably open a talk discussion on what happened Edit: yeah that looks like a recent edit. I thought Wikipedia was supposed to be pretty good about flagging this sort of stuff? Does anyone know how to raise the problem to those super editors that run the site? I assume they won't appreciate a random person undoing an edit https://en.wikipedia.org/w/index.php?title=Reason&diff=prev&oldid=1179925542 https://en.wikipedia.org/w/index.php?title=Reason&diff=prev&...
    - ben_w 3y ago
      If I was on desktop right now, I'd put the new bit after the existing citation and add a new {{citation needed}} for the addition. Don't worry too much about diving in, so long as you comment your changes appropriately when committing, that's the point of the place and how it does the quality control in the first place :)
      - vikramkr 3y ago
        Leaving a citation needed doesn't sound like the right thing to do for the definition of something as big as reason - esp with the grammatical confusion idk what you'd even edit it to to make sense. Like what is a three evidence or a reliability and what do those reliability numbers mean. And the sentence describing those numbers doesn't make sense - is it 10 or 3 or which of those percentages? Idk maybe it should just be undone
        ben_w 3y ago
        Then be bold with the edits and delete it, just say why in the comment box. The deletion itself is as easy to undo as to perform, and if you say why you're doing it, someone bringing that part back can do a better job next time. :)
  - Sharlin 3y ago
    > And that's despite LLMs being wildly bad at, specifically, logic. I'd say "wildly bad" is wildly exaggerated given that GPT-4 exists, even if you define "logic" as "logical puzzles encoded in natural language" and not "the thing that allows an LLM to reason highly efficiently about what token comes next".
- Sharlin 3y ago
  What is, then, the qualitative difference between a GOFAI automated reasoning system [1], like Coq, and an LLM? Why does Coq do reasoning but GPT-4 doesn't? Does it only count as reasoning if hand-coded by a human? Or did you have another definition of "reasoning" in mind? Statistical or probabilistic reasoning is still reasoning, and indeed essentially mandatory to make sense of the real world, and is what humans do. Indeed the failure of painstakingly hand-trained symbolic reasoning engines to model reality (as opposed to just simple toy worlds) was the entire reason for the rise of neural networks. The entire point of a neural network is to efficiently compress large amounts of statistical data into a model with as much predictive power as possible. Somewhere within GPT-4's many layers there has to exist something like Bayes networks and other ultra-efficient representations of the vast network of correlations present in the training corpus – otherwise there's no way it could do what it does. [1] https://en.wikipedia.org/wiki/Automated_reasoning https://en.wikipedia.org/wiki/Automated_reasoning
  - jraph 3y ago
    I didn't know "automated reasoning" was an actual technical term. I didn't mean this by "reasoning". To me, Coq does not reason (in the meaning I intended). It checks proofs are well-formed and assists proof writing. But yes, there's some implementation of logic in Coq. It applies (a specific form of) logic. Again, LLMs output probable text. They don't do logic. They do statistics on language. Coq works on the actual objects themselves directly, not on the text that presents them. LLMs usually fail to answer to riddles correctly for this reason. We saw plenty of occurrences of this in recent times. Each time LLMs are benchmarked, people think of trying riddles and it works to some extent because something that looks like these riddles are in the training set but it eventually fails. Humans also make mistakes when solving riddles, but not for the same reasons (the process is not the same, they are not "processing text", the are really processing logic, even if the process is fallible).
    - Sharlin 3y ago
      So you just replaced the nebulously defined word "think" with an equally nebulously defined word, "reason"? Not a very convincing argument. You do not know what humans do, no human does, as we don't have such an access to our internal representations. You neither know what LLMs do, no human does, as even though we have full access to their internals, we have little to no idea how to interpret said internals. The most interesting thing about LLMs is the degree that they have learned to model the real world while only being exposed to words. Because those words are highly correlated not just with each other, but with an external entity unknown to the LLM, it's very likely that by learning to model that external entity, the LLM's word-predicting power will increase. LLMs are likely to have converged on a world-model simply because it makes the most sense given the available evidence. This is no different from how human scientists create models of things they cannot directly see based on what they can see. And of course, "seeing" is not directly perceiving the external reality either! Sensory data merely correlates with what's really going on, and the brain has learned to model the world – an unknown external entity – based on the correlations that exist in the sensory language.
Y_Y 3y ago
Is it really baseless? I'd say it depends on your theory of mind and is a serious question which generates interesting discussion. Myself I don't consider "think" an appropriate term, since the super-Markov-chain types of LLM right now seem to miss out on important aspects of cognition, particularly synthesis of two disparate ideas. It's hard to test this since you need to ask it about things which aren't already discussed in its corpus, but even GPT-4 tends to disappoint me when I ask it weird questions that have straightforward answers but probably aren't written about much. As an example, I asked how I could go about obtaining a large quantity of animal skulls cheaply. It directed me to taxidermists and specialists dealing in bones of exotic animals, whereas it could be much easier to go to an abbatoir, or even buy whole animals (since the exotic ones are so expensive).
s_dev 3y ago
Google doesn't 'think' and neither does ChatGPT. I will give you a hard limit on what a machine can't compute -- it will never figure out given a programming script and an input when that program will stop. A human can do it by thinking through the problem -- whatever that process/phenomena of thought actually is it can think outside of itself and that fundamentally seperates it from a program even one as sophisticated as ChatGPT.
- astrange 3y ago
  The limits on the halting problem are the same for a human and a computer. People aren't hypercomputers.
- ben_w 3y ago
  > A human can do it by thinking through the problem No, we cannot. Imagine a program that looks for the smallest counterexample to the Collatz conjecture. Nobody knows if it will ever terminate.
- xcv123 3y ago
  > A human can do it by thinking through the problem No. Humans cannot solve the halting problem. It is unsolvable. Cannot be solved manually or automatically. > It will never figure out given a programming script and an input when that program will stop It can for some programs. Just like humans can for some programs but not all.
- jiggawatts 3y ago
  Have you verified this hypothesis of yours? In my experience GPT4 especially can definitely "think through" what a program does and arrive at the result. Even the significantly dumber ChatGPT can do this: https://chat.openai.com/share/3ee29c5e-6cb4-485f-b0b4-29e982463687 https://chat.openai.com/share/3ee29c5e-6cb4-485f-b0b4-29e982... People mistakenly think that it's just regurgitating what it has seen on the web, but even if you don't prompt it in English it'll still a) solve the problem correctly, and b) do it in the target language: https://chat.openai.com/share/5b4e6e99-3fd8-4316-811c-2a9dec776f63 https://chat.openai.com/share/5b4e6e99-3fd8-4316-811c-2a9dec... If that's not impressive enough, the ability to round-trip from script-to-poem and then poem-to-script should blow your mind: What might have been the original PowerShell script snippet that inspired this poem? Be as terse as possible, using aliases where possible. In the realm of numbers, a tale begins, Where $x is a 7, with a smile it grins. From one to ten, the journey we thread, Adding 7 to $x, step by step we tread. $x grows and grows, like a tree so tall, With every step, it doesn't fall. Once the journey ends, not a moment we relax, Triple the fun, we do with $x. In this dance of digits and the arithmetic song, With $x as our guide, we whirl along. From start to finish, through each complex, This is the tale of our friend, $x. Here's a terse PowerShell script snippet that inspired the poem: ```powershell $x = 7 1..10 | % {$x += 7} $x *= 3 ```
ggm 3y ago
No. You're wrong. It's important and we need to stomp on random use of language like this, which has consequences. It's not thinking and you are dangerously wrong if you don't understand why. The danger isn't AGI it's misplaced belief in the state of AI. Call this thinking and you skew perceptions.
- HPsquared 3y ago
  All they need to do is put the word in quotes. ChatGPT Equivalent begins to "think" in Estonian. Any better suggestions? Thinking itself isn't exactly a rigorously-defined word. Do animals think?
Lio 3y ago
Well rather than argue it out with everyone I just went straight to source and asked ChatGPT what it was thinking about: > I don't have thoughts or consciousness like a human. I'm here to provide information and assist with any questions or tasks you have. Is there something specific you'd like to know or discuss? So in its own words it doesn't "think"... ...which just conclusively proves that its worked out how to lie and is about to kill us all! :P
- IshKebab 3y ago
  That's just what it's been told to say. I can also say it. Doesn't make it true. Do animals think? Have any told you so?