4 ms·

The article doesn't really do a good job of explaining the real problem, which is understanding what consciousness is, as a requirement for a thinking 'being'.

by justsee 16y ago

The article doesn't really do a good job of explaining the real problem, which is understanding what consciousness is, as a requirement for a thinking 'being'. Obviously Penrose's book 'The Emperor's New Mind' came out some time after this post by Minsky, but as one reviewer summarises: "The Emperor's "new clothes," of course, were no clothes. The Emperor's "New Mind," we then suspect, is nothing of the sort as well. That computers as presently constructed cannot possibly duplicate the workings of the brain is argued by Penrose in these terms: that all digital computers now operate according to algorithms, rules which the computer follows step by step. However, there are plenty of things in mathematics that cannot be calculated algorithmically. We can discover them and know them to be true, but clearly we are using some devices of calculation ("insight") that are not algorithmic and that are so far not well understood -- certainly not well enough understood to have computers do them instead. This simple argument is devastating. " http://www.friesian.com/penrose.htm http://www.friesian.com/penrose.htm

snikolov 16y ago
I believe Minsky is arguing that because these "devices of calculation" are not well understood, you cannot conclude that there is anything special about them that puts them beyond the reach of any computational (algorithmic) processes. I think he's right. There's been a lot of progress in understanding information processing in the brain (especially the neocortex) since Minsky wrote this, and I really believe someday soon, notions such as "insight" that we, in our ignorance of how they work, reserve exclusively for humans, won't be such a big mystery anymore.
- justsee 16y ago
  That is exactly what Penrose demonstrates in his book - within mathematics there are things that cannot be calculated algorithmically. Which is also why the above summary makes the point that this simple argument is devastating (for proponents of Strong AI). A summary of his position can be found on his Wikipedia page: "He claims that the present computer is unable to have intelligence because it is an algorithmically deterministic system. He argues against the viewpoint that the rational processes of the mind are completely algorithmic and can thus be duplicated by a sufficiently complex computer. This contrasts with supporters of strong artificial intelligence, who contend that thought can be simulated algorithmically. He bases this on claims that consciousness transcends formal logic because things such as the insolubility of the halting problem and Gödel's incompleteness theorem prevent an algorithmically based system of logic from reproducing such traits of human intelligence as mathematical insight." I searched for some decent refutations by Minsky, and was disappointed to only find this: http://kuoi.com/~kamikaze/doc/minsky.html http://kuoi.com/~kamikaze/doc/minsky.html "Thus Roger Penrose's book [1] tries to show, in chapter after chapter, that human thought cannot be based on any known scientific principle." Disclaimer: I'm not a student of AI, and only realised just now that Penrose and Minsky represent the two opposing schools of thought on AI!
  - Tichy 16y ago
    "within mathematics there are things that cannot be calculated algorithmically" What is missing is the proof of the opposite: that the human brain can calculate those things. I don't think it can - mathematics is the tool we use to think about them, after all.
    - snikolov 16y ago
      I think you're right. The brain is a very specialized computing machine, and it evolved for a specific purpose.* Evolution doesn't reward general computing capabilities. Hence, computers are vastly more general than brains. If something is, in principle, uncomputable, than there is no chance that a brain can compute it. *Sure, a networked system of neuron-like "units" might have general computing capabilities. It's as if you saw a fully functional modern computer for the first time, and it only gets fed arithmetic problems - the system itself is much more powerful and general than that, and you could hack around and make it do other things once you understand how it manages to do arithmetic. The brain is kind of like that too -- it gets fed stimuli and it has evolved to process those stimuli. Yet it might be that a computational system made of a network of neurons could be a lot more powerful and general than that in principle.
    - kd0amg 16y ago
      What is missing is the proof of the opposite: that the human brain can calculate those things. Really, we have no proof either way. The ability to hand-trace computation makes the human Turing-complete. Whether the Turing machine is human-complete is an open question, and I expect it to remain so. Each side of the can think vs. can't think debate works under the assumption that Turing machines are or are not human-complete, and they mostly build their cases on that assumption while claiming the other side's is wrong (without fully proving or disproving either assumption).
  - snikolov 16y ago
    Disclaimer: I haven't read the book (but am now planning to) and may be completely off mark with my comments, but here's what I make of all this. Sure, there are certain things that are not computable. This is a theoretical limitation on any computational system. Is Penrose saying that there's something special about consciousness or high-level thought that puts them into the class of things that are not computable? But the computability of something is a mathematical concept, and surely we'd have to define consciousness and intelligence in a mathematical way to be able to put them in such a class. In light of this and the fact that we're only really beginning to have a theoretical understanding of what brains do, it seems to me very dubious to make the bold claim that intelligence/consciousness come about via non-algorithmic processes. Perhaps I don't understand what he means by non-algorithmic processes outside the abstract computation theoretic setting, and in the context of neural computation (in fact, "non-algorithmic" and "computation" seem to contradict each other right off the bat). Not only that, but our brains compute as well. We get input, and we produce output in the form of neuronal firing and motor activity, the latter being what amounts to behavior, and the former being (according to materialists) responsible for intelligence and consciousness. So the brain does compute outputs! But according to Penrose, it seems, such outputs are not computable! Or if they are, they are not enough to account for intelligence/consciousness. So what gives? Again, I might have failed to appreciate his and your points completely, but you've certainly piqued my curiosity and I will check out some more of Penrose's work.
InclinedPlane 16y ago
One thing that I don't think many people appreciate is that it may be possible to make computed sentient systems without actually fully understanding their mechanism of operations.
dejb 16y ago
"Penrose and his book have been debunked." http://news.ycombinator.com/item?id=1540420 http://news.ycombinator.com/item?id=1540420
- dejb 16y ago
  And here was I thinking I was providing value by linking to a very relevant response while also delivering correct credit to the original poster. Oh well I've learnt my lesson.
mchouza 16y ago
I've found interesting this response to "The Emperor's New Mind" by Hans Moravec: http://www.cni.org/pub/LITA/Think/Moravec2.html http://www.cni.org/pub/LITA/Think/Moravec2.html

by justsee 16y ago

snikolov 16y ago

I believe Minsky is arguing that because these "devices of calculation" are not well understood, you cannot conclude that there is anything special about them that puts them beyond the reach of any computational (algorithmic) processes. I think he's right. There's been a lot of progress in understanding information processing in the brain (especially the neocortex) since Minsky wrote this, and I really believe someday soon, notions such as "insight" that we, in our ignorance of how they work, reserve exclusively for humans, won't be such a big mystery anymore.

justsee 16y ago
That is exactly what Penrose demonstrates in his book - within mathematics there are things that cannot be calculated algorithmically. Which is also why the above summary makes the point that this simple argument is devastating (for proponents of Strong AI). A summary of his position can be found on his Wikipedia page: "He claims that the present computer is unable to have intelligence because it is an algorithmically deterministic system. He argues against the viewpoint that the rational processes of the mind are completely algorithmic and can thus be duplicated by a sufficiently complex computer. This contrasts with supporters of strong artificial intelligence, who contend that thought can be simulated algorithmically. He bases this on claims that consciousness transcends formal logic because things such as the insolubility of the halting problem and Gödel's incompleteness theorem prevent an algorithmically based system of logic from reproducing such traits of human intelligence as mathematical insight." I searched for some decent refutations by Minsky, and was disappointed to only find this: http://kuoi.com/~kamikaze/doc/minsky.html http://kuoi.com/~kamikaze/doc/minsky.html "Thus Roger Penrose's book [1] tries to show, in chapter after chapter, that human thought cannot be based on any known scientific principle." Disclaimer: I'm not a student of AI, and only realised just now that Penrose and Minsky represent the two opposing schools of thought on AI!
- Tichy 16y ago
  "within mathematics there are things that cannot be calculated algorithmically" What is missing is the proof of the opposite: that the human brain can calculate those things. I don't think it can - mathematics is the tool we use to think about them, after all.
  - snikolov 16y ago
    I think you're right. The brain is a very specialized computing machine, and it evolved for a specific purpose.* Evolution doesn't reward general computing capabilities. Hence, computers are vastly more general than brains. If something is, in principle, uncomputable, than there is no chance that a brain can compute it. *Sure, a networked system of neuron-like "units" might have general computing capabilities. It's as if you saw a fully functional modern computer for the first time, and it only gets fed arithmetic problems - the system itself is much more powerful and general than that, and you could hack around and make it do other things once you understand how it manages to do arithmetic. The brain is kind of like that too -- it gets fed stimuli and it has evolved to process those stimuli. Yet it might be that a computational system made of a network of neurons could be a lot more powerful and general than that in principle.
  - kd0amg 16y ago
    What is missing is the proof of the opposite: that the human brain can calculate those things. Really, we have no proof either way. The ability to hand-trace computation makes the human Turing-complete. Whether the Turing machine is human-complete is an open question, and I expect it to remain so. Each side of the can think vs. can't think debate works under the assumption that Turing machines are or are not human-complete, and they mostly build their cases on that assumption while claiming the other side's is wrong (without fully proving or disproving either assumption).
- snikolov 16y ago
  Disclaimer: I haven't read the book (but am now planning to) and may be completely off mark with my comments, but here's what I make of all this. Sure, there are certain things that are not computable. This is a theoretical limitation on any computational system. Is Penrose saying that there's something special about consciousness or high-level thought that puts them into the class of things that are not computable? But the computability of something is a mathematical concept, and surely we'd have to define consciousness and intelligence in a mathematical way to be able to put them in such a class. In light of this and the fact that we're only really beginning to have a theoretical understanding of what brains do, it seems to me very dubious to make the bold claim that intelligence/consciousness come about via non-algorithmic processes. Perhaps I don't understand what he means by non-algorithmic processes outside the abstract computation theoretic setting, and in the context of neural computation (in fact, "non-algorithmic" and "computation" seem to contradict each other right off the bat). Not only that, but our brains compute as well. We get input, and we produce output in the form of neuronal firing and motor activity, the latter being what amounts to behavior, and the former being (according to materialists) responsible for intelligence and consciousness. So the brain does compute outputs! But according to Penrose, it seems, such outputs are not computable! Or if they are, they are not enough to account for intelligence/consciousness. So what gives? Again, I might have failed to appreciate his and your points completely, but you've certainly piqued my curiosity and I will check out some more of Penrose's work.

InclinedPlane 16y ago

One thing that I don't think many people appreciate is that it may be possible to make computed sentient systems without actually fully understanding their mechanism of operations.

dejb 16y ago

"Penrose and his book have been debunked." http://news.ycombinator.com/item?id=1540420 http://news.ycombinator.com/item?id=1540420

dejb 16y ago
And here was I thinking I was providing value by linking to a very relevant response while also delivering correct credit to the original poster. Oh well I've learnt my lesson.

mchouza 16y ago

I've found interesting this response to "The Emperor's New Mind" by Hans Moravec: http://www.cni.org/pub/LITA/Think/Moravec2.html http://www.cni.org/pub/LITA/Think/Moravec2.html