4 ms·

I can highly recommend nobel laureate Sir Roger Penroses' take on AI. He argues that AI does in no way 'undestand' anything. He goes even further and uses Goede

I can highly recommend nobel laureate Sir Roger Penroses' take on AI. He argues that AI does in no way 'undestand' anything. He goes even further and uses Goedels Theorem to show that 'understanding' itself is non-computeable. I am not sure about this, but after playing with chatGPT I can clearly see what he means by lack of 'understanding'. There is absolutely ZERO real understanding.... only correlations and probabilities of words....

jcims 4y ago
User idiotsecant posted this excerpt from ChatGPT a couple of weeks ago. It kind of blows my mind (the last paragraph). It feels a bit beyond just basic probabilities of words. https://news.ycombinator.com/item?id=33833178 https://news.ycombinator.com/item?id=33833178 We're just at the beginning.
- bratwurst3000 4y ago
  My opinion is as long the dont solve hard problems they are just probability . For example one of those top ten mathematical problems. Humans solved a few of those, lets challenge AI
  - gnomewrecker 4y ago
    By this metric the vast majority of humans are “just probability” too. I think it’s not a very good metric.
    - bratwurst3000 4y ago
      Hmm is see it this way. Humans are a network of information sharing that could solve this problems. Computers are the same and should also Solve hard problems. And yes, sadly, many humans are just probalility in my eyes.
- notahacker 4y ago
  Didn't blow mine, even though it involved a howitzer! It's obviously operating on a higher level than "what was the previous word" but doesn't exactly feel like high level reasoning: when prompted for similarities it elides "inflated" with "blew up" to seed a paragraph (rather than to crack a corny joke, which is the only context I can imagine a human in possession of knowledge of what a howitzer and a beach ball doing likewise!), adds some sentences consistent with sentences in its corpus with "blowing up" and "loading" as the respective verbs and appends a sentence which states that it's tenuous, because howitzer and beach balls and inflating and loading verbs aren't often found in close proximity in its corpus (and caveating stuff clearly was well rewarded in ChatGPT's training; it's almost comically quick to do so in every single non-trivial answer) Impressive that it does so in coherent English, but I'm not convinced it's in any danger of understanding firearms or fun beyond realising sentence forms those words would be a good fit and poor fit for. I posted an excerpt from ChatGPT a couple of days ago where it stated - in an identical format to its previous two correct answers - that 355 was a prime number because it wasn't divisible 5, and that's the sort of problem that Turing machines are known to be able to completely and accurately model
hexatin 4y ago
Forgive me if this is covered in his argument since I haven’t read it yet, but does he discuss why this same argument wouldn’t apply to biological brains?
- eigenspace 4y ago
  He thinks our biological brains get around it by using quantum computation. Roger Penrose is a historical hero of the physics community and a current embarrassment from time to time. His ideas on things like computation and neurobiology are not worth taking seriously. It’s unfortunate to see people like the one you’re responding to using his Nobel Laureate credentials to justify his nonsense.
bryan0 4y ago
What is understanding if not correlations of words and ideas and concepts? I totally respect Penrose but unless he has an noncomputable alternative for the brain then that’s just magic (and completely unnecessary). Now I know Penrose used to have ideas how the brain is actually a quantum computer which uses entanglement. But that is still computable.
soulofmischief 4y ago
So in other words, humans also have zero real understanding.
- Jensson 4y ago
  Humans add structure to unstructured data, the AI models we have just make flawed replications of structure we feed it, that is the difference. ChatGPT didn't even figure out the structure of basic math which is the most salient logical structure humans have. Even idiot humans can learn to count and compare quantities without being hardcoded to do so, they learn it from just words and pictures. A language model that failed to learn this when it was trained on the entire internet therefore can't have any understanding the way humans understands things, and feeding it more compute or data wont get it there either. Example of a question a typical human idiot can solve without ever getting it wrong, but ChatGPT can't reliably: Is 7 dollars enough to buy a thing costing 7 dollars? ChatGPT can usually solve this, but it sometimes gets it wrong, getting that sort of thing wrong ever means that it doesn't understand, it just uses dumb statistics. Edit: I'm not saying it is impossible to make AI models that understand these things, just that the ones we have today don't.
  - chatgptexpert 4y ago
    Dumb statistics you say? https://i.imgur.com/im9EquK.png https://i.imgur.com/im9EquK.png Perhaps it's magic trick that a language model can do math, but it's quite impressive that it gets very close to the desired answer.
    - Jensson 4y ago
      It probably had problems with this part "two one dollar bills". A lot of text that contains "two" and "one" results in a "three", but also many texts with "two" and "one" results in "two", and then the model randomly chooses between those two interpretations. The worst part is that it doesn't even do it consistently for the same piece of text, creating that nonsense.
andyjohnson0 4y ago
Link? > He argues that AI does in no way 'undestand' anything. He goes even further and uses Goedels Theorem to show that 'understanding' itself is non-computeable. So how do brains understand things? Is he asserting that something is happening in the brain that can't be described/modelled computationally?
- anankaie 4y ago
  Yes. Penrose hypothesizes a non-computational (which would be interesting, because it opens a whole an of worms about how to model it) constituent of the universe. A little while ago he, along with Stuart Hammeroff, proposed the Orch-OR model, which posited that this is the same constituent as what causes the "collapse" of an unmeasured superposition of states into one concrete, measured state. With that said, I have not seen any of his work on this for some time (I last looked at it around 2016), though a cursory look shows some new papers of his on this and related subjects.
- astrange 4y ago
  Penrose's argument is dualism (humans have souls) with a lot of mathematical handwaving around it involving claims there are quantum computers in your brain. I do not see why it should be respected or why you should think he's different from Deepak Chopra.
AbrahamParangi 4y ago
The bar should not be “does the machine understand” but rather, “can the machine be distinguished from a typical human by a test” And further, for any test we choose we must ask ourselves, “do we expect our ability to discriminate between these two classes, man and machine, to go to zero as machine performance continues to improve” If we believe that only continued performance improvements are needed to make AI indistinguishable then that means today we’re only talking about differences in quality rather than differences in kind.
HarHarVeryFunny 4y ago
Why would you equate ChatGPT with AI ? It's just a language model. I know it's fashionable to call anything machine-learning related "AI", but it makes for non-sensical conversations such as here where Penrose is using "AI" to refer to any potential future AI (something that at least appears to have human level intelligence), but you're using "AI" to refer to something completely different - a dumb language model. Penrose is one of those people who is "religiously" against AI. He will clutch at any straw possible to argue that AI will never be conscious, or understand anything, or encroach on our human uniqueness and/or spirituality in any way. His favorite argument seems to be that our brain is utilizing (inherently non-computable - at least by non-quantum computer) quantum effects in the brain's microtubules, but I'm sure if it wasn't that it'd be something else. He doesn't WANT to believe, therefore he finds an excuse not to believe.
- janalsncm 4y ago
  Probably the main reason for this confusion is the lack of a good definition of “intelligence”. Basically no consensus if you browse the associated Wikipedia page. Of course the definition is also bogged down in human-centered history, assuming that humans are at the pinnacle of some contrived intelligence scale and placing other animals below us. I’m sure the genius philosophers who came up with this scale would’ve placed themselves at the pinnacle within humans as well. Tacking “artificial” on top of that doesn’t make it any clearer. My understanding of the taxonomy of these things within CS is that AI is a broad class of techniques for problem solving. Machine learning is a subset of those techniques which uses data and statistical methods. Non-ML AI is sometimes called GOFAI (“good old-fashioned AI”).
  - HarHarVeryFunny 4y ago
    AI and ML are two different things. AI can either be considered as a research area or simply end goal/result. AI doesn't itself refer to any set of techniques - it's the goal, not the means to the goal. You can consider GOFAI as non-ML, although perhaps it's more correct to regard it as older symbolic approaches (rule-based systems, etc) vs modern connectionist/ANN ones. Machine Learning (ML) is a catch-all term (not in of itself a technology) for any approach where systems are designed to learn from data. This includes techniques such as random forests and SVMs, as well as neural nets. There's really no fundamental relationship between AI and ML. Neither one is a subset of the other as this is an apples and oranges comparison - one is a goal, and the other is an approach). That said, all recent progress towards AI has been achieved by using ML, although not all uses of ML can really be regarded as AI. Hope that helps define terms.
jmfldn 4y ago
ChatGPT does not understand, I don't think there is any such thing as machine understanding nor will there ever be. At least not within the current paradigm of computing. My guess is that our kind of understanding requires a biological substrate and / or something quantum. Obviously I'm hand waving here as I'm out of my depth with this subject. My sense though is that there are things about the brain about its evolution that are eluding us, and which are significant. Here's the thing though. Even on the current trajectory, we will produce programs that sure as hell APPEAR like they understand. This is really what Alan Turing was getting at when he said that the question of whether a machine can think is meaningless. It's like asking whether a submarine can swim. The fact is, dead machine or not, the outputs will increasingly exhibit, what look like, emergent properties of understanding, and even apparent sentience. This to me is the truly stunning thing about ChatGPT. It might have limitations, but through mere statistics and 'auto complete on steroids' it can do way more than you'd think was possible with such an approach.
- Arjuna144 4y ago
  True, except for this: > Obviously I'm hand waving here as I'm out of my depth with this subject. My sense though is that there are things about the brain about its evolution that are eluding us, and which are significant. Please go and deepen you meditations. First: Get to the point, where you have stripped your awareness away from your sensory input (pratyahara) (which everyone may know, it can be done, when you are deeply concentrated, someone may have said something to you, but you didn't "hear" it) If you are at that point, your entire "experience" will consist of your "inner world" (thoughs and emotions). So next is to "concentrate" your mind (Dharana) by bringing it back to the same object (lets say a rose for example). This takes A LOT of practice but after the years, you mind will become clear and focused so that the time where only the object (the rose) is in your mind becomes longer and longer (a few seconds up to minutes). As you practice further episodes of undisturbed, completely one-pointed mind become longer and longer and you enter into Dhyana (meditation). You are completely focused on the object, and the object itself is all that you experience. Imagine what that means: your entire experience constist of only the rose and nothing but the rose!!! As this goes on and on for longer and longer periods of time, the yogis and mystics say that the boundaries between "you" the subject and "the rose" the object start to dissolve more and more, until they vanish. You are the rose and the rose is. Nothing but the rose that you are!! (They call it samadhi, the dissolution of individual existence and merging of object and subject). This is just on an object but when the object is your awareness itself and this process happens, they call it many different names depending on tradition. ("Nirvikalpa Samadhi" in the yogic tradition, "Unio-Mystica" in the christian traditions, "fana" or "Baqaa billah" in the islamic mystical tradition. It is the realization of the unity of consciousness that is and that only is and that "I" merges into...

jcims 4y ago

User idiotsecant posted this excerpt from ChatGPT a couple of weeks ago. It kind of blows my mind (the last paragraph). It feels a bit beyond just basic probabilities of words. https://news.ycombinator.com/item?id=33833178 https://news.ycombinator.com/item?id=33833178 We're just at the beginning.

bratwurst3000 4y ago
My opinion is as long the dont solve hard problems they are just probability . For example one of those top ten mathematical problems. Humans solved a few of those, lets challenge AI
- gnomewrecker 4y ago
  By this metric the vast majority of humans are “just probability” too. I think it’s not a very good metric.
  - bratwurst3000 4y ago
    Hmm is see it this way. Humans are a network of information sharing that could solve this problems. Computers are the same and should also Solve hard problems. And yes, sadly, many humans are just probalility in my eyes.
notahacker 4y ago
Didn't blow mine, even though it involved a howitzer! It's obviously operating on a higher level than "what was the previous word" but doesn't exactly feel like high level reasoning: when prompted for similarities it elides "inflated" with "blew up" to seed a paragraph (rather than to crack a corny joke, which is the only context I can imagine a human in possession of knowledge of what a howitzer and a beach ball doing likewise!), adds some sentences consistent with sentences in its corpus with "blowing up" and "loading" as the respective verbs and appends a sentence which states that it's tenuous, because howitzer and beach balls and inflating and loading verbs aren't often found in close proximity in its corpus (and caveating stuff clearly was well rewarded in ChatGPT's training; it's almost comically quick to do so in every single non-trivial answer) Impressive that it does so in coherent English, but I'm not convinced it's in any danger of understanding firearms or fun beyond realising sentence forms those words would be a good fit and poor fit for. I posted an excerpt from ChatGPT a couple of days ago where it stated - in an identical format to its previous two correct answers - that 355 was a prime number because it wasn't divisible 5, and that's the sort of problem that Turing machines are known to be able to completely and accurately model

hexatin 4y ago

Forgive me if this is covered in his argument since I haven’t read it yet, but does he discuss why this same argument wouldn’t apply to biological brains?

eigenspace 4y ago
He thinks our biological brains get around it by using quantum computation. Roger Penrose is a historical hero of the physics community and a current embarrassment from time to time. His ideas on things like computation and neurobiology are not worth taking seriously. It’s unfortunate to see people like the one you’re responding to using his Nobel Laureate credentials to justify his nonsense.

bryan0 4y ago

What is understanding if not correlations of words and ideas and concepts? I totally respect Penrose but unless he has an noncomputable alternative for the brain then that’s just magic (and completely unnecessary). Now I know Penrose used to have ideas how the brain is actually a quantum computer which uses entanglement. But that is still computable.

soulofmischief 4y ago

So in other words, humans also have zero real understanding.

Jensson 4y ago
Humans add structure to unstructured data, the AI models we have just make flawed replications of structure we feed it, that is the difference. ChatGPT didn't even figure out the structure of basic math which is the most salient logical structure humans have. Even idiot humans can learn to count and compare quantities without being hardcoded to do so, they learn it from just words and pictures. A language model that failed to learn this when it was trained on the entire internet therefore can't have any understanding the way humans understands things, and feeding it more compute or data wont get it there either. Example of a question a typical human idiot can solve without ever getting it wrong, but ChatGPT can't reliably: Is 7 dollars enough to buy a thing costing 7 dollars? ChatGPT can usually solve this, but it sometimes gets it wrong, getting that sort of thing wrong ever means that it doesn't understand, it just uses dumb statistics. Edit: I'm not saying it is impossible to make AI models that understand these things, just that the ones we have today don't.
- chatgptexpert 4y ago
  Dumb statistics you say? https://i.imgur.com/im9EquK.png https://i.imgur.com/im9EquK.png Perhaps it's magic trick that a language model can do math, but it's quite impressive that it gets very close to the desired answer.
  - Jensson 4y ago
    It probably had problems with this part "two one dollar bills". A lot of text that contains "two" and "one" results in a "three", but also many texts with "two" and "one" results in "two", and then the model randomly chooses between those two interpretations. The worst part is that it doesn't even do it consistently for the same piece of text, creating that nonsense.

andyjohnson0 4y ago

Link? > He argues that AI does in no way 'undestand' anything. He goes even further and uses Goedels Theorem to show that 'understanding' itself is non-computeable. So how do brains understand things? Is he asserting that something is happening in the brain that can't be described/modelled computationally?

anankaie 4y ago
Yes. Penrose hypothesizes a non-computational (which would be interesting, because it opens a whole an of worms about how to model it) constituent of the universe. A little while ago he, along with Stuart Hammeroff, proposed the Orch-OR model, which posited that this is the same constituent as what causes the "collapse" of an unmeasured superposition of states into one concrete, measured state. With that said, I have not seen any of his work on this for some time (I last looked at it around 2016), though a cursory look shows some new papers of his on this and related subjects.
astrange 4y ago
Penrose's argument is dualism (humans have souls) with a lot of mathematical handwaving around it involving claims there are quantum computers in your brain. I do not see why it should be respected or why you should think he's different from Deepak Chopra.

AbrahamParangi 4y ago

The bar should not be “does the machine understand” but rather, “can the machine be distinguished from a typical human by a test” And further, for any test we choose we must ask ourselves, “do we expect our ability to discriminate between these two classes, man and machine, to go to zero as machine performance continues to improve” If we believe that only continued performance improvements are needed to make AI indistinguishable then that means today we’re only talking about differences in quality rather than differences in kind.

HarHarVeryFunny 4y ago

Why would you equate ChatGPT with AI ? It's just a language model. I know it's fashionable to call anything machine-learning related "AI", but it makes for non-sensical conversations such as here where Penrose is using "AI" to refer to any potential future AI (something that at least appears to have human level intelligence), but you're using "AI" to refer to something completely different - a dumb language model. Penrose is one of those people who is "religiously" against AI. He will clutch at any straw possible to argue that AI will never be conscious, or understand anything, or encroach on our human uniqueness and/or spirituality in any way. His favorite argument seems to be that our brain is utilizing (inherently non-computable - at least by non-quantum computer) quantum effects in the brain's microtubules, but I'm sure if it wasn't that it'd be something else. He doesn't WANT to believe, therefore he finds an excuse not to believe.

janalsncm 4y ago
Probably the main reason for this confusion is the lack of a good definition of “intelligence”. Basically no consensus if you browse the associated Wikipedia page. Of course the definition is also bogged down in human-centered history, assuming that humans are at the pinnacle of some contrived intelligence scale and placing other animals below us. I’m sure the genius philosophers who came up with this scale would’ve placed themselves at the pinnacle within humans as well. Tacking “artificial” on top of that doesn’t make it any clearer. My understanding of the taxonomy of these things within CS is that AI is a broad class of techniques for problem solving. Machine learning is a subset of those techniques which uses data and statistical methods. Non-ML AI is sometimes called GOFAI (“good old-fashioned AI”).
- HarHarVeryFunny 4y ago
  AI and ML are two different things. AI can either be considered as a research area or simply end goal/result. AI doesn't itself refer to any set of techniques - it's the goal, not the means to the goal. You can consider GOFAI as non-ML, although perhaps it's more correct to regard it as older symbolic approaches (rule-based systems, etc) vs modern connectionist/ANN ones. Machine Learning (ML) is a catch-all term (not in of itself a technology) for any approach where systems are designed to learn from data. This includes techniques such as random forests and SVMs, as well as neural nets. There's really no fundamental relationship between AI and ML. Neither one is a subset of the other as this is an apples and oranges comparison - one is a goal, and the other is an approach). That said, all recent progress towards AI has been achieved by using ML, although not all uses of ML can really be regarded as AI. Hope that helps define terms.

jmfldn 4y ago

ChatGPT does not understand, I don't think there is any such thing as machine understanding nor will there ever be. At least not within the current paradigm of computing. My guess is that our kind of understanding requires a biological substrate and / or something quantum. Obviously I'm hand waving here as I'm out of my depth with this subject. My sense though is that there are things about the brain about its evolution that are eluding us, and which are significant. Here's the thing though. Even on the current trajectory, we will produce programs that sure as hell APPEAR like they understand. This is really what Alan Turing was getting at when he said that the question of whether a machine can think is meaningless. It's like asking whether a submarine can swim. The fact is, dead machine or not, the outputs will increasingly exhibit, what look like, emergent properties of understanding, and even apparent sentience. This to me is the truly stunning thing about ChatGPT. It might have limitations, but through mere statistics and 'auto complete on steroids' it can do way more than you'd think was possible with such an approach.

Arjuna144 4y ago
True, except for this: > Obviously I'm hand waving here as I'm out of my depth with this subject. My sense though is that there are things about the brain about its evolution that are eluding us, and which are significant. Please go and deepen you meditations. First: Get to the point, where you have stripped your awareness away from your sensory input (pratyahara) (which everyone may know, it can be done, when you are deeply concentrated, someone may have said something to you, but you didn't "hear" it) If you are at that point, your entire "experience" will consist of your "inner world" (thoughs and emotions). So next is to "concentrate" your mind (Dharana) by bringing it back to the same object (lets say a rose for example). This takes A LOT of practice but after the years, you mind will become clear and focused so that the time where only the object (the rose) is in your mind becomes longer and longer (a few seconds up to minutes). As you practice further episodes of undisturbed, completely one-pointed mind become longer and longer and you enter into Dhyana (meditation). You are completely focused on the object, and the object itself is all that you experience. Imagine what that means: your entire experience constist of only the rose and nothing but the rose!!! As this goes on and on for longer and longer periods of time, the yogis and mystics say that the boundaries between "you" the subject and "the rose" the object start to dissolve more and more, until they vanish. You are the rose and the rose is. Nothing but the rose that you are!! (They call it samadhi, the dissolution of individual existence and merging of object and subject). This is just on an object but when the object is your awareness itself and this process happens, they call it many different names depending on tradition. ("Nirvikalpa Samadhi" in the yogic tradition, "Unio-Mystica" in the christian traditions, "fana" or "Baqaa billah" in the islamic mystical tradition. It is the realization of the unity of consciousness that is and that only is and that "I" merges into...