13 ms·

Here is, to me, the relevant line: "Hubert Dreyfus, who argued that computers, who have no body, no childhood and no cultural practice, could not acquire intel

Here is, to me, the relevant line: "Hubert Dreyfus, who argued that computers, who have no body, no childhood and no cultural practice, could not acquire intelligence at all. One of Dreyfus’ main arguments was that human knowledge is partly tacit, and therefore cannot be articulated and incorporated in a computer program. " I have not read the rest of the article but in the introduction it's stated: "The article further argues that this is in principle impossible, and it revives Hubert Dreyfus’ argument that computers are not in the world." Wiktionary defines tacit as "Not derived from formal principles of reasoning" [1]. So the main argument is that humans have intelligence that is impossible to express through reason or codification. In other words, humans have a literal soul, divorced from physical world, that cannot be expressed in our physical world thus making any endeavour to create artificial intelligence impossible. This is a dualist line of reasoning and, in my opinion, is nothing more than theology dressed up in philosophy. I would much rather the author just flat out say they are a dualist or that they reject the Church-Turing thesis.

dynamite-ready 6y ago
I'm not sure. If we think about it mechanically, and the output of the a computer system is a function of it's input, then perhaps we are woefully underestimating the role of the body as both a monitor and input system.
- empath75 6y ago
  So build a body.
  - hnick 6y ago
    I don't see why they need the same 5 or more senses we have. Self-driving cars have rudimentary visual input, proximity sensors, location data, and other inputs.
    - empath75 6y ago
      Right. In what way is a human embodied but not a self driving car.
  - dynamite-ready 6y ago
    At this stage at least, the recreation of the interaction between that artificial body and the environment will be impressionistic at best... Like we don't even have a full conscious understanding of what we are made from, or what we need to survive. How do we 'install' those ideas and that imperative in an agent, when we don't fully understand it ourselves? I'm not entirely supporting one side or another, but I think it's reasonable to bet against the imminent arrival of AGI at this stage, unless some radical discovery comes to light soon. And if (when?) that discovery eventually comes, I'd suspect it to be a biological one. But then, who knows...
  - baddox 6y ago
    What is "a body" other than inputs and outputs to interface with the environment, and perhaps a mechanism of perceiving the environment and remembering what was perceived?
    - codr7 6y ago
      The capital T-truth is that no one really knows since we're all observing the system subjectively from the inside. Many who practice meditation and/or experiment with psychedelic drugs will tell you that there's something in there that's not a computer. We could assume, as many do, they're fooling themselves; since we can't measure it. Or, we could trust our authentic experience of the real world even when it can't (yet) be measured.
      - empath75 6y ago
        I had the opposite opinion after taking psychedelic drugs.
- flywheel 6y ago
  I don't think an artificial intelligence has to resemble anything we would recognize as human intelligence, while still being "intelligent". The motivations of humans and the motivations of computers are very different. If we can give a computer motivation, as well as a way to act and react within an environment, and a framework for learning (neural networks?) then what it "thinks" will be determined by the experience of its own existence. I'm not sure we're there yet.
mkl 6y ago
I don't think it has anything to do with souls. Most "knowledge" deep learning systems have seems tacit to me: it would be practically impossible for people to write programs that articulate and incorporate that knowledge without machine learning (people tried, for decades), and it certainly isn't "derived from formal principles of reasoning" in anything but a tangential mathematical sense. (I too have not read the whole article; I'm just replying to this comment.)
- baddox 6y ago
  > it would be practically impossible for people to write programs that articulate and incorporate that knowledge without machine learning That could be true, but I think that's a different argument. That's more like claiming that it is impossible for a computer to become intelligent unless it can experience its environment and remember those experiences. That seems like a much more plausible and less dualist claim.
- leereeves 6y ago
  I'd argue that deep learning is derived from formal principles of mathematical reasoning in a very concrete sense. Deep learning learns a predictive function of the features that minimizes the loss function (with some caveats). If the loss function and training data are well chosen, that minimizes the probability of being wrong.
- simonh 6y ago
  I have read it and you’re not missing anything. One of the examples of tacit knowledge they give is walking and that therefore it is impossible to teach a computer to walk. They should watch one of the videos from Boston Dynamics.
  - Noos 6y ago
    That's not teaching a computer to walk, that's building a walking machine. Subtle difference, but it's easy to prove; no matter how well you build a Boston dynamics robot, it will never like walking in the same way your TV will never like entertaining people. There's no "I" there to learn.
SiVal 6y ago
Tacit knowledge is knowledge that results from adapting to experience, like learning to tie shoelaces with practice, rather than something like finding the derivative of sin(x) by the usual mathematical proof method (reasoning step by formal step). Every deep learning system has tacit knowledge: it knows a chair when it sees one but can't explain how it knows. It just adapted its connections to training until it got it right most of the time. So computers are provably capable of what, in humans, is defined as tacit knowledge and can be given sensors and actuators to learn from. A car can learn to parallel park with practice. It can't explain how it does it, but you can copy the trained system into a new car. I don't see why you couldn't produce a combination of sensors and actuators that vastly exceeds what any human is capable of. But AGI isn't that. It's a variety of information processing techniques (algorithms) deployed as a toolbox managed by meta-techniques (more algorithms) that know how to deploy the others in various combinations. We don't yet know much about the management algorithms, but I don't see any reason in principle why we couldn't eventually find some and invent others.
- yehbut99 6y ago
  Computers can’t heal themselves. Our bodies do that on their own without conscious prompting. Humans evolve the complexity of a computers electron states, not the computers own inherent properties. My use doesn’t force transistors to evolve into better transistors. It has no self regeneration. You don’t see the issue in principle but here you have an article by an expert pointing them out. Perhaps be a better listener? A computer has observable, literal limitations relative to a humans mechanical functionality. You can’t scrape away literal reality to arrive at some reductionist idea of what a consciousness is. Our only known good model for a machine that can create our consciousness is us. It took billions of years of the universe churning at random to accidentally generate us. We have no clue how to replicate that scale. A computer literally lacks a whole lot of literal information that’s embedded in the hardware and software of a person. Watch this and tell me the last time your computer reconfigured it’s literal shape when you altered its electric field properties: https://youtu.be/RjD1aLm4Thg https://youtu.be/RjD1aLm4Thg There’s something to “life” we’ll never be able to jam into silicon.
  - 6y ago
forgotmypw17 6y ago
I'm not sure how you can say computers have no body and no childhood. I think someone who doesn't know computers very well just doesn't see it, but they actually have both.
- wwwwewwww 6y ago
  I think I know computers well. But I have no idea what "body" and "childhood" computers have. Can you please explain?
  - simonh 6y ago
    Their body is their hardware, sensors, interfaces and means of changing the world through controlled devices which can include robot arms and legs or vehicles. Computers can learn and be trained to do things and get better at them, such as using genetic algorithms and machine learning. They’re rudimentary, but real.
    - ddingus 6y ago
      They aren't of sufficient complexity and robustness to self identify. Our bodies offer a closed loop, very robust and complex feedback system. Our intelligence can process that and learn all sorts of things. Can learn what is "them" and what is other. Without those things, a computer, as we have constructed them so far, really don't have a childhood.
      - forgotmypw17 6y ago
        I think if you perceive a computer as a single cell of an organism, you can see more clearly what I mean. And computers' childhood is happening right now.
        ddingus 6y ago
        That's more like evolution, like basic chemicals combining. Computers are not yet like organisms. They are getting super complex. A childhood is a very different thing still.
coldtea 6y ago
>So the main argument is that humans have intelligence that is impossible to express through reason or codification. In other words, humans have a literal soul, divorced from physical world No, in other words, humans have tactile, empirical, emotional, social, etc intelligence that is perfectly physical but not available to mere software in a computer. It might be available to software running in humanoid robots, that can see, walk around, hang with other humans to learn, etc. But even in that case, it won't be codified in any axiomatic way "through reason". Think more of neural networks and less of an 1960s AI program...
- cageface 6y ago
  This seems like a weak argument to me. We don't really understand how human intelligence works yet so how can we claim that computers will never realize similar intelligence? We don't know for a fact that human intelligence depends on these things. I'm personally skeptical that we'll see AGI any time soon but I don't think we know enough to say this definitively.
  - coldtea 6y ago
    >This seems like a weak argument to me. We don't really understand how human intelligence works yet so how can we claim that computers will never realize similar intelligence? My comment doesn't say that "computers will never realize similar intelligence". It says that they will never realize it through reasoning - and rule based systems, 1960s-1990s AI style. Which isn't the way we realize it either, even if we don't fully understand how we do realize it yet.
- ddingus 6y ago
  One thing humans, and living things in general have, is enough complexity to be self aware on some level. There is US, our parts, and the rest of the world, and OTHERS. A computer, brain in a box, lacks that. Maybe one embodied in such a way as to be capable of "the self" may display emergent properties similar to how we and the animals do.
  - 6y ago
mongojunction 6y ago
We think things that aren't computable. Probabilistic quantum nature of superposition. But I don't think that means that "Turing machines" can't have AGI. What I think what the article is talking about, is the idea that AGI must be embodied. That a widely unrecognized aspect of HI is its embodiment (read The Power of not thinking), and that without a body machines cannot be in the world, and therefore cannot have AGI. I tend to think however that machine AGI will be possible even without putting them in robot bodies. They already have their own world, the internet. Just get them to explore that with a virtual analogy of a body. Worth a try anyway.
contravariant 6y ago
That's a bit of a straw-man. It does not follow that the existence of knowledge that isn't derived from formal principles of reasoning proves the existence of a soul.
- simonh 6y ago
  Replace the word soul with some essential quality of reason than humans have that isn’t teachable, and the article is arguing that whatever that is can never be acquired by a computer.
acituan 6y ago
You are taking huge liberties in forming equivocations which mislead your conclusion. The tacit knowledge description only talks about the acquisition process of the knowledge, not that the knowledge itself is outside bounds of rationality or the physical world. By the same token when you claim "humans have intelligence that is impossible to express through reason or codification" the entire argument hinges on the actual meaning of impossible. Is it impossible in itself, or ever, because human intelligence is non-reason based or are we using a meaning of impossible that at this day and age the process of doing that is still intractable for all intents and purposes. If you're claiming the former, that itself is such a strong claim it requires its own strong proof. If it's the latter, well we have been developing psychotechnologies for several millennia to be able to express ourselves and our cognitive processes and getting better at it, you just need to be patient. If you found a working x86 chip in the wild, but all documentation and knowledgeable people were wiped out for some reason, I bet the process of finding out how that x86 worked would look quite similar. It wouldn't make x86 otherworldly or in need of a soul.
- biswaroop 6y ago
  This is a good point, but I will add, from close discussions with Dreyfus, his position was that it's impossible because it's fundamentally impossible, not because it's intractable. (see my other comment for more)
  - visarga 6y ago
    Interesting to learn about Dreyfus, I was aware of this line of thought from Francois Chollet's article. He is the creator of Keras and some pretty advanced research papers in the nature of intelligence. He states that the environment and embodiment are crucial for the development of intelligence. Even for humans, 'our environment puts a hard limit on our individual intelligence'. https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec https://medium.com/@francois.chollet/the-impossibility-of-in... The implausibility of intelligence explosion (2017) .. In essence we need simulators on par with reality to train human-like intelligence. BTW, take a look at ThreeDWorld, just came out: 'A High-Fidelity, Multi-Modal Platform for Interactive Physical Simulation'. We're getting closer, and AI scientists are aware of the environment problem. http://www.threedworld.org/ http://www.threedworld.org/
biswaroop 6y ago
I have a perspective on this, since I took a rather engaging philosophy class from the late Prof Dreyfus at Berkeley. As the article hints at, his line of argument was a completely valid criticism of AI based on a set of rules written with symbolic logic. He arrived at this conclusion after studying Heidegger's concept of dasein. The best way to describe dasein is through a classic example [1]: "...the hammer is involved in an act of hammering; that hammering is involved in making something fast; and that making something fast is involved in protecting the human agent against bad weather. Such totalities of involvements are the contexts of everyday equipmental practice. As such, they define equipmental entities, so the hammer is intelligible as what it is only with respect to the shelter and, indeed, all the other items of equipment to which it meaningfully relates in Dasein's everyday practices. " In Heidegger's mind, meaning is distributed across the web of interrelationships between objects and their various uses and ideas to humans. Intelligence was thus a process of knowing those relationships, and being a part of them. Being-in-the-world (dasein is a German word related to 'being') is a result of us as humans being 'thrown' into the web of meaning, in fact we are born already finding ourselves in it. Dreyfus' objections to AI stemmed from the idea that computers are thrown into the world differently from us. They are the hammer in the anecdote, and not the human. During the class, we argued a lot with him about whether humans are the only creatures that have 'being-in-the-world'. We asked about the idea of the soul, and why humans are unique in this paradigm. My feeling after the end was that his entire philosophy comes directly from Heidegger. And since Heidegger didn't mention animals, or the lack of souls, his conclusion was "dunno". Related to this, Dreyfus had an understanding of physics that was quite antiquated and classical. One example was the concept of time. According to Dreyfus, the physicists' notion of time was a series of discrete observations along a timeline, whereas humans experience time in stretches and long segments. I remember telling him that the notion of a single instant of time is poorly defined in quantum physics, and that we do recognize that every event has some time uncertainty. I remember asking him why we couldn't model human experience with some complicated function. For him, everything came back to Heidegger. To him, we physicists were being too reductionist, and there is indeed something about humans that cannot be described using physics. He stopped shy of calling it a soul, but it was essentially that. [ETA: I now remember talking to him about Conway's game of life, and how a simple set of rules could result in complex, often hard to model behavior. My point was emergent systems exist within physics, and the way we describe them is different from single particles. His reply suggested that he was certain that human experience wasn't just 'emergent' - it was fundamentally different from anything in physics, no matter what.] The class was on physics vs philosophy, and we disagreed. I don't think he really understood what I was trying to tell him. This was in 2013, before most of the deep learning revolution, but I think he would have the same objections with what we have now. Here are two possible directions we can go from here: 1) Argue with continental philosophers about reductionism and whether humans have a unique essence that cannot be modeled. 2) Understand Heidegger's and Dreyfus' thoughts about being and time, and drive AI research in better directions. I prefer (2), because I already tried (1) with Dreyfus and it wasn't successful or productive. I think understanding and modeling the graph of inter-relationships between objects and humans is exactly where we need to improve when it comes to AGI. It's probably going to need a good degree of embodiedness, an idea of a computer being thrown-into-the-world. Dreyfus would tell us that it's fundamentally impossible, and that we should all just give up on it. I think he came to the wrong conclusion. What I get from Heidegger is not "AGI is impossible" but rather "hey, this is what we should be worrying about. This is how humans see the world". TLDR: Dreyfus had some good ideas about AI. I think it's extremely insightful to pay attention his thoughts. Don't waste time worrying about his arguments against physicalism. [1] https://plato.stanford.edu/entries/heidegger/ https://plato.stanford.edu/entries/heidegger/
kevin_thibedeau 6y ago
Sociopaths can pretend to be like normal people. An AI with non-human intelligence can do its own thing and just mimic human behavior with sufficient fidelity to fool us. Dumb machines can already employ usable conversational interfaces. In 100 years, less dumb machines will be far more convincing.
yters 6y ago
I don't think it is dualist, or at least I didn't get that impression reading Dreyfus. It is more that knowledge is part of a system as a whole and cannot be captured piecemeal, which is how we'd have to do it if we created AI by hand. But, I don't think that argument is especially strong. Knowledge doesn't seem to be tied up in the system, otherwise we wouldn't have abstract subjects like math. Additionally, if knowledge is a function of a certain process of developmwnt, we can at least in theory reproduce any physical process computationally.
pier25 6y ago
> So the main argument is that humans have intelligence that is impossible to express through reason or codification I think what it means is not about codifcation, but that to achieve GI machines would need to go through human experiences that are not possible to a machine. I don't agree, but I think that's the point the paper makes.
User23 6y ago
The Church-Turing thesis only states that the lambda calculus and Turing machines can compute the same functions. It has nothing to do with materialism or dualism and it certainly doesn’t state that the universe is a Turing machine.
- deleuze 6y ago
  It's funny, because I'd argue that the belief that "the universe is a Turing machine" is a kind of secular religion. Hardcore physicalists often betray themselves as not only being bad materialists but actually just idealists in denial.
  - User23 6y ago
    I have to confess I take some pleasure in asking people that say we're living in a simulation "oh, you're a creationist?"
    - webmaven 6y ago
      An amusing retort, to be sure, but the "creator" could be an entirely automated process, akin to instantiating a VM or container. In other words, God has been a replaced with a very small shell script. So, even if an intelligent creator is ultimately responsible for our plane of existence, there may not be much in the way of intent or even observation associated with that responsibility at any scale we would find meaningful. Heck, who is to say that our particular simulated universe isn't just a honeypot of some sort?
- abetusk 6y ago
  The Church-Turing thesis is: "Every effectively calculable function is a computable function" [1] The definition is a little terse so we have to expand on what "effectively calculable function" and "computable function" mean. By a "computable function", we mean a Turing machine. The term "effectively calculable function" is a little unclear but one definition that I think is the closest to the intent is "it can be done by a human without any aids except writing materials." [2]. In other words, the Church-Turing thesis is saying: "All physically computable functions are Turing computable" That is, the physical world, including human cognition, can be realized by a Turing machine. While one formulation might be cast in terms of lambda calculus, this is hiding the underlying assumption that lambda calculus is used as a proxy to simulation of the physical world and, as a subset, human cognition, effectively saying "if it can do lambda calculus, it can do the physical universe and can do human cognition". [1] https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#Statement_in_Church's_and_Turing's_words https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#S... [2] https://en.wikipedia.org/wiki/Effective_method#Definition https://en.wikipedia.org/wiki/Effective_method#Definition
nl 6y ago
The author of the article specifically says this: I have earlier said that neural networks need not be programmed, and therefore can handle tacit knowledge. Also you are misunderstanding what tacit means. It doesn't mean anything mystical - merely that it is gained from observations rather than logically reasoning about something.
- abetusk 6y ago
  You might be right but I'm not inclined to give the author the benefit of the doubt. From the abstract, the author clearly says: "The article further argues that ... computers are not in the world." The specific quote you mention is in a larger paragraph which says: "Computers are not in our world. I have earlier said that neural networks need not be programmed, and therefore can handle tacit knowledge. However, it is simply not true, as some of the advocates of Big Data argue, that the data “speak for themselves”. Normally, the data used are related to one or more models, they are selected by humans, and in the end they consist of numbers." My reading of this is that "tacit" is used as a kind of dog whistle to dualists. It's ambiguous enough so that the author can claim they meant "learned" while still suggesting an underlying dualism. Regardless of the meaning of "tacit" in this context, the author pretty much flat out says they're a dualist by repeatedly claiming "computers are not in our world" and "in the end they consist of numbers". I think you're giving the author too much credibility that they aren't making a plea to mysticism.
  - nl 6y ago
    (I've read the whole article.) I don't think the author makes a strong argument, and I disagree with their conclusion. But the author's argument actually appears to be mostly that science itself is insufficient to understand the world. This argument is outlined in the section starting "But the replacement of our everyday world by the world of science is based on a fundamental misunderstanding. Edmund Husserl was one of the first who pointed this out, and attributed this misunderstanding to Galileo." I think it's a pretty weak argument and I'm surprised Nature published it - the author wouldn't last 2 minutes trying to defend it on HN. I do think that a better articulated version of his argument would be something like this (which attempts to capture what he means by "not in the world"): "despite all the advances in neural networks encoding tacit knowledge, it still takes a deeper human-level set of tacit knowledge in a wider context to make these neural networks useful. While we have surpassed human skills in-the-small, science based benchmarks, we seem no closer to achieving embodied human-level intelligence from machines in-the-large." But tacit still doesn't mean what you claimed.
deleted 6y ago
[deleted]
visarga 6y ago
> So the main argument is that humans have intelligence that is impossible to express through reason or codification. In other words, humans have a literal soul, divorced from physical world, that cannot be expressed in our physical world thus making any endeavour to create artificial intelligence impossible. No, it just means there is no simple symbolic path that is human understandable towards human level intelligence. We need systems like neural nets and simulators to 'learn' things that can't be directly formalised. And we have tried for 50 years to formalise intelligence in symbolic representations.
- glenstein 6y ago
  >there is no simple symbolic path that is human understandable towards human level intelligence I imagine that, piece by piece, if we really wanted to, we could look at the 175 billion parameters in GPT-3, test which are 'active' when this poem is written, which are active when that imitation of copypasta is active, and through a torturous interrogation, find that one particular parameter, say, is for weighting the 0.001% likelihood that you would use an accented é during a particular rhetorical flourish in certain contexts. Perhaps another parameter codes a meta-meta-meta abstraction about how a meta-meta rule governs a meta-rule for how to use a sometimes-used linguistic rule. And the totality of those 175 billion parameters could in principle be uncovered and described in ways that are satisfactory to humans. It would be tedious and unproductive, and akin to the project of archeologists patiently, tediously uncovering a dinosaur. But the point is it would be practically difficult, not something forbidden as a matter of principle. More importantly though, is that I don't think the supposed incomprehensibility to humans has relevance to anything. What's the argument supposed to be? Humans don't depend in any explicit, conscious way, on having conscious grasp of our own tacit knowledge. I don't know why I unconsciously shift my weight a certain way when going up stairs. This doesn't stop me from walking up stairs. And it doesn't stop us from making machines that could walk up stairs. We need neural nets? Okay, sure, we need them. But we can run those on machines that, at the end of the day, are silicon and 0s and 1s, which are every bit the brute, formal systems that supposedly can't model intelligent things. Weren't we supposed to have encountered a barrier to what computers can do at some point in this thought exercise? Because it appears that what began as an extremely bold claim, that computers can't do X, Y, and Z, ends in a whimper, as a vague exhortation to appreciate that neural nets are in some sense structurally different from logic design. Nothing about that latter claim is making any bold statements about the limits of what computers can or can't do, which makes me feel like the argument forgot what it was supposed to be about halfway through.
friendlybus 6y ago
You don't need a soul to justify the existence of knowledge that can't be reasoned. Qualia, emotion, feeling and knowledge soley from sensory observation fill that gap too. All of these things are perceived as a meaning by our brains well before they are rationalized. Feelings and emotions in particular break down under direct reasoned examination. The mission to simulate a human mind seems more like a cultural precept for programmers. The physical task to create a human mind simulation is a fool's errand. The human body is incredibly complex and I doubt we'll stumble over the ability to sythensize that system any time this century or next.
proc0 6y ago
Yes, completely agree and was thinking along these lines as I read. I'm really interested in a good argument against AGI. Turing had the right intuition in thinking that if our brains are processing information then they can be simulated by a Turing machine. Everything points to our brains doing information processing at all levels. The article still has a point about general intelligence without a body, but I think this can be solved by developing robotics and AI together, and I've seen some research in this area (I think Japan).
kitd 6y ago
If by "soul" you mean an emotional core, then you'd be correct. Emotions allow us to short-circuit the processing required to ascribe value to a thing or situation. But that has nothing to do with dualism or theology.
badrabbit 6y ago
Why does general AI have to be like human intelligence? Free will might have something to do with it as well. Seld awareness is the basis of our desire to find our place in the world and the meaning of reality relative to self. This desire might be a core driver of human intelligence, the intial lack of purpose,self awareness and the need to survive by solving arbitrary problems is important as well. Computers may not live in the real world but a virtual world of games, coupled with self aware program that trains subprograms to adopt and solve solutions to survive might be an interesting approach.

dynamite-ready 6y ago

I'm not sure. If we think about it mechanically, and the output of the a computer system is a function of it's input, then perhaps we are woefully underestimating the role of the body as both a monitor and input system.

empath75 6y ago
So build a body.
- hnick 6y ago
  I don't see why they need the same 5 or more senses we have. Self-driving cars have rudimentary visual input, proximity sensors, location data, and other inputs.
  - empath75 6y ago
    Right. In what way is a human embodied but not a self driving car.
- dynamite-ready 6y ago
  At this stage at least, the recreation of the interaction between that artificial body and the environment will be impressionistic at best... Like we don't even have a full conscious understanding of what we are made from, or what we need to survive. How do we 'install' those ideas and that imperative in an agent, when we don't fully understand it ourselves? I'm not entirely supporting one side or another, but I think it's reasonable to bet against the imminent arrival of AGI at this stage, unless some radical discovery comes to light soon. And if (when?) that discovery eventually comes, I'd suspect it to be a biological one. But then, who knows...
- baddox 6y ago
  What is "a body" other than inputs and outputs to interface with the environment, and perhaps a mechanism of perceiving the environment and remembering what was perceived?
  - codr7 6y ago
    The capital T-truth is that no one really knows since we're all observing the system subjectively from the inside. Many who practice meditation and/or experiment with psychedelic drugs will tell you that there's something in there that's not a computer. We could assume, as many do, they're fooling themselves; since we can't measure it. Or, we could trust our authentic experience of the real world even when it can't (yet) be measured.
    - empath75 6y ago
      I had the opposite opinion after taking psychedelic drugs.
flywheel 6y ago
I don't think an artificial intelligence has to resemble anything we would recognize as human intelligence, while still being "intelligent". The motivations of humans and the motivations of computers are very different. If we can give a computer motivation, as well as a way to act and react within an environment, and a framework for learning (neural networks?) then what it "thinks" will be determined by the experience of its own existence. I'm not sure we're there yet.

mkl 6y ago

I don't think it has anything to do with souls. Most "knowledge" deep learning systems have seems tacit to me: it would be practically impossible for people to write programs that articulate and incorporate that knowledge without machine learning (people tried, for decades), and it certainly isn't "derived from formal principles of reasoning" in anything but a tangential mathematical sense. (I too have not read the whole article; I'm just replying to this comment.)

baddox 6y ago
> it would be practically impossible for people to write programs that articulate and incorporate that knowledge without machine learning That could be true, but I think that's a different argument. That's more like claiming that it is impossible for a computer to become intelligent unless it can experience its environment and remember those experiences. That seems like a much more plausible and less dualist claim.
leereeves 6y ago
I'd argue that deep learning is derived from formal principles of mathematical reasoning in a very concrete sense. Deep learning learns a predictive function of the features that minimizes the loss function (with some caveats). If the loss function and training data are well chosen, that minimizes the probability of being wrong.
simonh 6y ago
I have read it and you’re not missing anything. One of the examples of tacit knowledge they give is walking and that therefore it is impossible to teach a computer to walk. They should watch one of the videos from Boston Dynamics.
- Noos 6y ago
  That's not teaching a computer to walk, that's building a walking machine. Subtle difference, but it's easy to prove; no matter how well you build a Boston dynamics robot, it will never like walking in the same way your TV will never like entertaining people. There's no "I" there to learn.

SiVal 6y ago

Tacit knowledge is knowledge that results from adapting to experience, like learning to tie shoelaces with practice, rather than something like finding the derivative of sin(x) by the usual mathematical proof method (reasoning step by formal step). Every deep learning system has tacit knowledge: it knows a chair when it sees one but can't explain how it knows. It just adapted its connections to training until it got it right most of the time. So computers are provably capable of what, in humans, is defined as tacit knowledge and can be given sensors and actuators to learn from. A car can learn to parallel park with practice. It can't explain how it does it, but you can copy the trained system into a new car. I don't see why you couldn't produce a combination of sensors and actuators that vastly exceeds what any human is capable of. But AGI isn't that. It's a variety of information processing techniques (algorithms) deployed as a toolbox managed by meta-techniques (more algorithms) that know how to deploy the others in various combinations. We don't yet know much about the management algorithms, but I don't see any reason in principle why we couldn't eventually find some and invent others.

yehbut99 6y ago
Computers can’t heal themselves. Our bodies do that on their own without conscious prompting. Humans evolve the complexity of a computers electron states, not the computers own inherent properties. My use doesn’t force transistors to evolve into better transistors. It has no self regeneration. You don’t see the issue in principle but here you have an article by an expert pointing them out. Perhaps be a better listener? A computer has observable, literal limitations relative to a humans mechanical functionality. You can’t scrape away literal reality to arrive at some reductionist idea of what a consciousness is. Our only known good model for a machine that can create our consciousness is us. It took billions of years of the universe churning at random to accidentally generate us. We have no clue how to replicate that scale. A computer literally lacks a whole lot of literal information that’s embedded in the hardware and software of a person. Watch this and tell me the last time your computer reconfigured it’s literal shape when you altered its electric field properties: https://youtu.be/RjD1aLm4Thg https://youtu.be/RjD1aLm4Thg There’s something to “life” we’ll never be able to jam into silicon.
- 6y ago

forgotmypw17 6y ago

I'm not sure how you can say computers have no body and no childhood. I think someone who doesn't know computers very well just doesn't see it, but they actually have both.

wwwwewwww 6y ago
I think I know computers well. But I have no idea what "body" and "childhood" computers have. Can you please explain?
- simonh 6y ago
  Their body is their hardware, sensors, interfaces and means of changing the world through controlled devices which can include robot arms and legs or vehicles. Computers can learn and be trained to do things and get better at them, such as using genetic algorithms and machine learning. They’re rudimentary, but real.
  - ddingus 6y ago
    They aren't of sufficient complexity and robustness to self identify. Our bodies offer a closed loop, very robust and complex feedback system. Our intelligence can process that and learn all sorts of things. Can learn what is "them" and what is other. Without those things, a computer, as we have constructed them so far, really don't have a childhood.
    - forgotmypw17 6y ago
      I think if you perceive a computer as a single cell of an organism, you can see more clearly what I mean. And computers' childhood is happening right now.
      - ddingus 6y ago
        That's more like evolution, like basic chemicals combining. Computers are not yet like organisms. They are getting super complex. A childhood is a very different thing still.

coldtea 6y ago

>So the main argument is that humans have intelligence that is impossible to express through reason or codification. In other words, humans have a literal soul, divorced from physical world No, in other words, humans have tactile, empirical, emotional, social, etc intelligence that is perfectly physical but not available to mere software in a computer. It might be available to software running in humanoid robots, that can see, walk around, hang with other humans to learn, etc. But even in that case, it won't be codified in any axiomatic way "through reason". Think more of neural networks and less of an 1960s AI program...

cageface 6y ago
This seems like a weak argument to me. We don't really understand how human intelligence works yet so how can we claim that computers will never realize similar intelligence? We don't know for a fact that human intelligence depends on these things. I'm personally skeptical that we'll see AGI any time soon but I don't think we know enough to say this definitively.
- coldtea 6y ago
  >This seems like a weak argument to me. We don't really understand how human intelligence works yet so how can we claim that computers will never realize similar intelligence? My comment doesn't say that "computers will never realize similar intelligence". It says that they will never realize it through reasoning - and rule based systems, 1960s-1990s AI style. Which isn't the way we realize it either, even if we don't fully understand how we do realize it yet.
ddingus 6y ago
One thing humans, and living things in general have, is enough complexity to be self aware on some level. There is US, our parts, and the rest of the world, and OTHERS. A computer, brain in a box, lacks that. Maybe one embodied in such a way as to be capable of "the self" may display emergent properties similar to how we and the animals do.
- 6y ago

mongojunction 6y ago

We think things that aren't computable. Probabilistic quantum nature of superposition. But I don't think that means that "Turing machines" can't have AGI. What I think what the article is talking about, is the idea that AGI must be embodied. That a widely unrecognized aspect of HI is its embodiment (read The Power of not thinking), and that without a body machines cannot be in the world, and therefore cannot have AGI. I tend to think however that machine AGI will be possible even without putting them in robot bodies. They already have their own world, the internet. Just get them to explore that with a virtual analogy of a body. Worth a try anyway.

contravariant 6y ago

That's a bit of a straw-man. It does not follow that the existence of knowledge that isn't derived from formal principles of reasoning proves the existence of a soul.

simonh 6y ago
Replace the word soul with some essential quality of reason than humans have that isn’t teachable, and the article is arguing that whatever that is can never be acquired by a computer.

acituan 6y ago

You are taking huge liberties in forming equivocations which mislead your conclusion. The tacit knowledge description only talks about the acquisition process of the knowledge, not that the knowledge itself is outside bounds of rationality or the physical world. By the same token when you claim "humans have intelligence that is impossible to express through reason or codification" the entire argument hinges on the actual meaning of impossible. Is it impossible in itself, or ever, because human intelligence is non-reason based or are we using a meaning of impossible that at this day and age the process of doing that is still intractable for all intents and purposes. If you're claiming the former, that itself is such a strong claim it requires its own strong proof. If it's the latter, well we have been developing psychotechnologies for several millennia to be able to express ourselves and our cognitive processes and getting better at it, you just need to be patient. If you found a working x86 chip in the wild, but all documentation and knowledgeable people were wiped out for some reason, I bet the process of finding out how that x86 worked would look quite similar. It wouldn't make x86 otherworldly or in need of a soul.

biswaroop 6y ago
This is a good point, but I will add, from close discussions with Dreyfus, his position was that it's impossible because it's fundamentally impossible, not because it's intractable. (see my other comment for more)
- visarga 6y ago
  Interesting to learn about Dreyfus, I was aware of this line of thought from Francois Chollet's article. He is the creator of Keras and some pretty advanced research papers in the nature of intelligence. He states that the environment and embodiment are crucial for the development of intelligence. Even for humans, 'our environment puts a hard limit on our individual intelligence'. https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec https://medium.com/@francois.chollet/the-impossibility-of-in... The implausibility of intelligence explosion (2017) .. In essence we need simulators on par with reality to train human-like intelligence. BTW, take a look at ThreeDWorld, just came out: 'A High-Fidelity, Multi-Modal Platform for Interactive Physical Simulation'. We're getting closer, and AI scientists are aware of the environment problem. http://www.threedworld.org/ http://www.threedworld.org/

biswaroop 6y ago

I have a perspective on this, since I took a rather engaging philosophy class from the late Prof Dreyfus at Berkeley. As the article hints at, his line of argument was a completely valid criticism of AI based on a set of rules written with symbolic logic. He arrived at this conclusion after studying Heidegger's concept of dasein. The best way to describe dasein is through a classic example [1]: "...the hammer is involved in an act of hammering; that hammering is involved in making something fast; and that making something fast is involved in protecting the human agent against bad weather. Such totalities of involvements are the contexts of everyday equipmental practice. As such, they define equipmental entities, so the hammer is intelligible as what it is only with respect to the shelter and, indeed, all the other items of equipment to which it meaningfully relates in Dasein's everyday practices. " In Heidegger's mind, meaning is distributed across the web of interrelationships between objects and their various uses and ideas to humans. Intelligence was thus a process of knowing those relationships, and being a part of them. Being-in-the-world (dasein is a German word related to 'being') is a result of us as humans being 'thrown' into the web of meaning, in fact we are born already finding ourselves in it. Dreyfus' objections to AI stemmed from the idea that computers are thrown into the world differently from us. They are the hammer in the anecdote, and not the human. During the class, we argued a lot with him about whether humans are the only creatures that have 'being-in-the-world'. We asked about the idea of the soul, and why humans are unique in this paradigm. My feeling after the end was that his entire philosophy comes directly from Heidegger. And since Heidegger didn't mention animals, or the lack of souls, his conclusion was "dunno". Related to this, Dreyfus had an understanding of physics that was quite antiquated and classical. One example was the concept of time. According to Dreyfus, the physicists' notion of time was a series of discrete observations along a timeline, whereas humans experience time in stretches and long segments. I remember telling him that the notion of a single instant of time is poorly defined in quantum physics, and that we do recognize that every event has some time uncertainty. I remember asking him why we couldn't model human experience with some complicated function. For him, everything came back to Heidegger. To him, we physicists were being too reductionist, and there is indeed something about humans that cannot be described using physics. He stopped shy of calling it a soul, but it was essentially that. [ETA: I now remember talking to him about Conway's game of life, and how a simple set of rules could result in complex, often hard to model behavior. My point was emergent systems exist within physics, and the way we describe them is different from single particles. His reply suggested that he was certain that human experience wasn't just 'emergent' - it was fundamentally different from anything in physics, no matter what.] The class was on physics vs philosophy, and we disagreed. I don't think he really understood what I was trying to tell him. This was in 2013, before most of the deep learning revolution, but I think he would have the same objections with what we have now. Here are two possible directions we can go from here: 1) Argue with continental philosophers about reductionism and whether humans have a unique essence that cannot be modeled. 2) Understand Heidegger's and Dreyfus' thoughts about being and time, and drive AI research in better directions. I prefer (2), because I already tried (1) with Dreyfus and it wasn't successful or productive. I think understanding and modeling the graph of inter-relationships between objects and humans is exactly where we need to improve when it comes to AGI. It's probably going to need a good degree of embodiedness, an idea of a computer being thrown-into-the-world. Dreyfus would tell us that it's fundamentally impossible, and that we should all just give up on it. I think he came to the wrong conclusion. What I get from Heidegger is not "AGI is impossible" but rather "hey, this is what we should be worrying about. This is how humans see the world". TLDR: Dreyfus had some good ideas about AI. I think it's extremely insightful to pay attention his thoughts. Don't waste time worrying about his arguments against physicalism. [1] https://plato.stanford.edu/entries/heidegger/ https://plato.stanford.edu/entries/heidegger/

kevin_thibedeau 6y ago

Sociopaths can pretend to be like normal people. An AI with non-human intelligence can do its own thing and just mimic human behavior with sufficient fidelity to fool us. Dumb machines can already employ usable conversational interfaces. In 100 years, less dumb machines will be far more convincing.

yters 6y ago

I don't think it is dualist, or at least I didn't get that impression reading Dreyfus. It is more that knowledge is part of a system as a whole and cannot be captured piecemeal, which is how we'd have to do it if we created AI by hand. But, I don't think that argument is especially strong. Knowledge doesn't seem to be tied up in the system, otherwise we wouldn't have abstract subjects like math. Additionally, if knowledge is a function of a certain process of developmwnt, we can at least in theory reproduce any physical process computationally.

pier25 6y ago

> So the main argument is that humans have intelligence that is impossible to express through reason or codification I think what it means is not about codifcation, but that to achieve GI machines would need to go through human experiences that are not possible to a machine. I don't agree, but I think that's the point the paper makes.

User23 6y ago

The Church-Turing thesis only states that the lambda calculus and Turing machines can compute the same functions. It has nothing to do with materialism or dualism and it certainly doesn’t state that the universe is a Turing machine.

deleuze 6y ago
It's funny, because I'd argue that the belief that "the universe is a Turing machine" is a kind of secular religion. Hardcore physicalists often betray themselves as not only being bad materialists but actually just idealists in denial.
- User23 6y ago
  I have to confess I take some pleasure in asking people that say we're living in a simulation "oh, you're a creationist?"
  - webmaven 6y ago
    An amusing retort, to be sure, but the "creator" could be an entirely automated process, akin to instantiating a VM or container. In other words, God has been a replaced with a very small shell script. So, even if an intelligent creator is ultimately responsible for our plane of existence, there may not be much in the way of intent or even observation associated with that responsibility at any scale we would find meaningful. Heck, who is to say that our particular simulated universe isn't just a honeypot of some sort?
abetusk 6y ago
The Church-Turing thesis is: "Every effectively calculable function is a computable function" [1] The definition is a little terse so we have to expand on what "effectively calculable function" and "computable function" mean. By a "computable function", we mean a Turing machine. The term "effectively calculable function" is a little unclear but one definition that I think is the closest to the intent is "it can be done by a human without any aids except writing materials." [2]. In other words, the Church-Turing thesis is saying: "All physically computable functions are Turing computable" That is, the physical world, including human cognition, can be realized by a Turing machine. While one formulation might be cast in terms of lambda calculus, this is hiding the underlying assumption that lambda calculus is used as a proxy to simulation of the physical world and, as a subset, human cognition, effectively saying "if it can do lambda calculus, it can do the physical universe and can do human cognition". [1] https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#Statement_in_Church's_and_Turing's_words https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#S... [2] https://en.wikipedia.org/wiki/Effective_method#Definition https://en.wikipedia.org/wiki/Effective_method#Definition

nl 6y ago

The author of the article specifically says this: I have earlier said that neural networks need not be programmed, and therefore can handle tacit knowledge. Also you are misunderstanding what tacit means. It doesn't mean anything mystical - merely that it is gained from observations rather than logically reasoning about something.

abetusk 6y ago
You might be right but I'm not inclined to give the author the benefit of the doubt. From the abstract, the author clearly says: "The article further argues that ... computers are not in the world." The specific quote you mention is in a larger paragraph which says: "Computers are not in our world. I have earlier said that neural networks need not be programmed, and therefore can handle tacit knowledge. However, it is simply not true, as some of the advocates of Big Data argue, that the data “speak for themselves”. Normally, the data used are related to one or more models, they are selected by humans, and in the end they consist of numbers." My reading of this is that "tacit" is used as a kind of dog whistle to dualists. It's ambiguous enough so that the author can claim they meant "learned" while still suggesting an underlying dualism. Regardless of the meaning of "tacit" in this context, the author pretty much flat out says they're a dualist by repeatedly claiming "computers are not in our world" and "in the end they consist of numbers". I think you're giving the author too much credibility that they aren't making a plea to mysticism.
- nl 6y ago
  (I've read the whole article.) I don't think the author makes a strong argument, and I disagree with their conclusion. But the author's argument actually appears to be mostly that science itself is insufficient to understand the world. This argument is outlined in the section starting "But the replacement of our everyday world by the world of science is based on a fundamental misunderstanding. Edmund Husserl was one of the first who pointed this out, and attributed this misunderstanding to Galileo." I think it's a pretty weak argument and I'm surprised Nature published it - the author wouldn't last 2 minutes trying to defend it on HN. I do think that a better articulated version of his argument would be something like this (which attempts to capture what he means by "not in the world"): "despite all the advances in neural networks encoding tacit knowledge, it still takes a deeper human-level set of tacit knowledge in a wider context to make these neural networks useful. While we have surpassed human skills in-the-small, science based benchmarks, we seem no closer to achieving embodied human-level intelligence from machines in-the-large." But tacit still doesn't mean what you claimed.

deleted 6y ago

[deleted]

visarga 6y ago

> So the main argument is that humans have intelligence that is impossible to express through reason or codification. In other words, humans have a literal soul, divorced from physical world, that cannot be expressed in our physical world thus making any endeavour to create artificial intelligence impossible. No, it just means there is no simple symbolic path that is human understandable towards human level intelligence. We need systems like neural nets and simulators to 'learn' things that can't be directly formalised. And we have tried for 50 years to formalise intelligence in symbolic representations.

glenstein 6y ago
>there is no simple symbolic path that is human understandable towards human level intelligence I imagine that, piece by piece, if we really wanted to, we could look at the 175 billion parameters in GPT-3, test which are 'active' when this poem is written, which are active when that imitation of copypasta is active, and through a torturous interrogation, find that one particular parameter, say, is for weighting the 0.001% likelihood that you would use an accented é during a particular rhetorical flourish in certain contexts. Perhaps another parameter codes a meta-meta-meta abstraction about how a meta-meta rule governs a meta-rule for how to use a sometimes-used linguistic rule. And the totality of those 175 billion parameters could in principle be uncovered and described in ways that are satisfactory to humans. It would be tedious and unproductive, and akin to the project of archeologists patiently, tediously uncovering a dinosaur. But the point is it would be practically difficult, not something forbidden as a matter of principle. More importantly though, is that I don't think the supposed incomprehensibility to humans has relevance to anything. What's the argument supposed to be? Humans don't depend in any explicit, conscious way, on having conscious grasp of our own tacit knowledge. I don't know why I unconsciously shift my weight a certain way when going up stairs. This doesn't stop me from walking up stairs. And it doesn't stop us from making machines that could walk up stairs. We need neural nets? Okay, sure, we need them. But we can run those on machines that, at the end of the day, are silicon and 0s and 1s, which are every bit the brute, formal systems that supposedly can't model intelligent things. Weren't we supposed to have encountered a barrier to what computers can do at some point in this thought exercise? Because it appears that what began as an extremely bold claim, that computers can't do X, Y, and Z, ends in a whimper, as a vague exhortation to appreciate that neural nets are in some sense structurally different from logic design. Nothing about that latter claim is making any bold statements about the limits of what computers can or can't do, which makes me feel like the argument forgot what it was supposed to be about halfway through.

friendlybus 6y ago

You don't need a soul to justify the existence of knowledge that can't be reasoned. Qualia, emotion, feeling and knowledge soley from sensory observation fill that gap too. All of these things are perceived as a meaning by our brains well before they are rationalized. Feelings and emotions in particular break down under direct reasoned examination. The mission to simulate a human mind seems more like a cultural precept for programmers. The physical task to create a human mind simulation is a fool's errand. The human body is incredibly complex and I doubt we'll stumble over the ability to sythensize that system any time this century or next.

proc0 6y ago

Yes, completely agree and was thinking along these lines as I read. I'm really interested in a good argument against AGI. Turing had the right intuition in thinking that if our brains are processing information then they can be simulated by a Turing machine. Everything points to our brains doing information processing at all levels. The article still has a point about general intelligence without a body, but I think this can be solved by developing robotics and AI together, and I've seen some research in this area (I think Japan).

kitd 6y ago

If by "soul" you mean an emotional core, then you'd be correct. Emotions allow us to short-circuit the processing required to ascribe value to a thing or situation. But that has nothing to do with dualism or theology.

badrabbit 6y ago

Why does general AI have to be like human intelligence? Free will might have something to do with it as well. Seld awareness is the basis of our desire to find our place in the world and the meaning of reality relative to self. This desire might be a core driver of human intelligence, the intial lack of purpose,self awareness and the need to survive by solving arbitrary problems is important as well. Computers may not live in the real world but a virtual world of games, coupled with self aware program that trains subprograms to adopt and solve solutions to survive might be an interesting approach.