6 ms·

> Seems pretty clear that we're on a path to machine intelligence I'm not sure what you mean by "machine intelligence," but there is absolutely no indication t

> Seems pretty clear that we're on a path to machine intelligence I'm not sure what you mean by "machine intelligence," but there is absolutely no indication that Strong AI can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible. Machines that learn, sure, but conscious machines, no way. Your droids, we don't serve their kind here. They'll have to wait outside. We don't want them here.

Udo 4y ago
What's your objection to "conscious" machines? (Putting it in quotes because that's a loaded, unscientific term and everybody has their own definition)
- Maursault 4y ago
  My objection is it would be cruel and horrifying. Also, as of yet, we don't understand what mind is, and we can slice up the brain in a thousand ways and never find it. We may never know, and all we do know is that consciousness is a phenomenon arising from healthy brain. So how are we supposed to make a machine conscious without knowing how we ourselves are conscious? Dr. John Searle's objection is that software is not a mind, and goes on to explain why with his Chinese Room thought experiment.[1] [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
  - Udo 4y ago
    How do you know you're not interacting with a Chinese Room when you talk to another human being? Our own brains are an excellent proof of concept that machines can be made to think. We're all machines in a way, after all - unless you believe there is a special supernatural exception for human minds.
    - Maursault 4y ago
      Let's not again confuse natural intelligence with artificial intelligence. You do not have artificial consciousness, you have genuine consciousness. Further, the brain is not a machine. To explain why, I'm just going to quote this excellent piece from mindmatters: A machine is an artifact. It is a human-built assembly of materials that have no natural inclination to work in unison. Silicon and copper, plastic and glass have no natural propensity to compute. We make artifacts, for example computers, out of these materials to serve our purposes. But they are not the materials’ natural purposes. In the classical hylemorphic (Aristotelian) perspective, an artifact is an artificial composite of substances and thus it has an accidental, not substantial, form. Machines work as they do “accidentally”; that is, their function as machines is not inherent to the matter that comprises them. Rather their function is imposed on the disparate parts by human intelligence. Unlike a machine, the brain is an organ, a functional part of a living organism. It (along with the body) has a substantial form; its activity is natural to it. One might say that the brain's activity is intrinsic to it, not imposed on it extrinsically, as is the activity of a machine.[1] [1] https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-if-you-choose-to-see-it-that-way/ https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-i...
      - TeMPOraL 4y ago
        The brain absolutely is a machine. Just like everything biological in general: life is nanotech. We know that, because we can literally look through our microscopes and see how it ticks, at the atomic level. So we looked and saw that life is just molecular nanotech, but what we didn't see was any tags attached to carbon chains that would say, "this molecule's natural function is to think", or "that molecule's natural function is to be a part of an organ that pumps blood", etc. Because there is no such thing like a "natural function" in the sense used by your quote, the distinction between "natural" and "artificial" is itself artificial[0], and that whole quote represents a piece of magical thinking philosophy you'd think humanity abandoned at least a century ago. ---- [0] - https://abstrusegoose.com/215 https://abstrusegoose.com/215
        Maursault 4y ago
        Words do have meanings, and "artificial" has a distinct meaning that is not the same meaning as that of "natural." Flavors natural and artificial are distinct from each other. There are natural functions. Pumping and circulating blood is a natural function of the heart. If you don't like "artificial," we can use "synthetic" or substitute "man-made." You have repeated assertions without any supporting argument. "The brain is absolutely a machine," is an assertion, but you have not remotely supported it. Proof by assertion[1] is fallacious argument. [1] https://en.wikipedia.org/wiki/Proof_by_assertion https://en.wikipedia.org/wiki/Proof_by_assertion
  - eru 4y ago
    > My objection is it would be cruel and horrifying. That's an argument for why we _shouldn't_ build them. But not an argument for why we _can't_ build them. Do you see the difference? > So how are we supposed to make a machine conscious without knowing how we ourselves are conscious? Evolution didn't 'know' about conscious either. The pioneers of the Industrial Revolution had only the faintest ideas about thermodynamics, yet managed to build working steam engines. (And we improved them over time, as we learned more.) For the 'Chinese Room' see eg https://www.scottaaronson.com/democritus/lec4.html https://www.scottaaronson.com/democritus/lec4.html or just have a look at good old Wikipedia: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room Though I'm not sure why you brought up consciousness in the first place. It's irrelevant for the Fermi Paradox.
    - Maursault 4y ago
      > That's an argument for why we _shouldn't_ build them. But not an argument for why we _can't_ build them. Do you see the difference? This is a straw man fallacy, because I was asked "What's your objection to "conscious" machines?", and my objection is that it is cruel. > Evolution didn't 'know' about conscious either. The pioneers of the Industrial Revolution had only the faintest ideas about thermodynamics, yet managed to build working steam engines. (And we improved them over time, as we learned more.) This is also a fallacious argument. Just because we don't know how evolution created consciousness doesn't mean that we can use our ignorance of how consciousness evolved to create machine consciousness. If you're only using this as evidence that consciousness is possible, the fallacy is begging the question, petitio principii. Also, I believe there is here equivocation and conflation of artificial intelligence with natural intelligence. If we have the time, we can spend 600M years to evolve consciousness directly, but then it would not be artificial, it would not be machine intelligence, it would be natural or biological intelligence, and it would be a massive waste of time because it is already here and we are it. > For the 'Chinese Room' see eg https://www.scottaaronson.com/democritus/lec4.html https://www.scottaaronson.com/democritus/lec4.html or just have a look at good old Wikipedia: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room Do you really expect me to argue against your citations when you're too lazy even to summarize any particular counter-argument? It has been said that most of the work done in AI has been in trying to prove John Searle wrong, which suggests his argument is significant. A Turing Test does not prove consciousness nor even intelligence. All it proves is whether a human can be fooled. Arguments suggesting emergent properties between a human and a rule book are patently ridiculous, absurd on their face, because learning a book could not possibly ever make a book conscious. Where the Chinese Room thought experiment breaks down is there is a human involved, and most counter-arguments get tripped up here forgetting that it is metaphor and there is no human inside a computer where these types of counter-arguments would ever come true: no matter how much machine learning you submit your program to, there is no mind to begin with that could ever become aware. > Though I'm not sure why you brought up consciousness in the first place. It's irrelevant for the Fermi Paradox. In fact, I did not, the OP of the tread asserted: "Seems pretty clear that we're on a path to machine intelligence," and I responded to that specifically and argued against the possibility.
  - nullc 4y ago
    > Dr. John Searle's objection is that software is not a mind, and goes on to explain why with his Chinese Room thought experiment. I always found that experiment to be extremely unconvincing, to put it mildly. Using that framework I can prove that you are not conscious. I construct a "english room". You are in the back without any contact to the outside world. A Chinese speaker in the front can pass messages to you from the outside world, in English by laborious copying. Clearly you aren't conscious because the Chinese speaker can't understand the English. Or, alternatively, no one would dispute a single neuron is not conscious and can't understand english. I pass signals into your neurons and through some electrochemical magic you reply in english, yet the neurons involved don't understand english therefor you aren't conscious. All Searle has done is picked out one component of a composite system, concluded that in isolation it has no clue whats going on, and from that concluded the system as a whole doesn't. We have a hard time clearly defining consciousness, and it's not clear if some AGI would be conscious (maybe it's just an emergent phenomena)-- it's also not clear if there would be any particular advantage in it being so. But in any case, I don't think the chinese room is a useful argument for anything.
jimmcslim 4y ago
> Machines that learn, sure, but conscious machines, no way. So what is the source of consciousness then? If it is something supernatural, like a soul, then yes reproducing consciousness in a machine is impossible. But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? I'm not sure there is a third possibility?
- Maursault 4y ago
  > So what is the source of consciousness then? Healthy brain is the source of consciousness. > But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? Though commonly referred to as such in metaphor, computers are not brains. Computers also have hard limitations that are very different from the limitations of biological brains. Maybe it is possible to accurately simulate a healthy brain but is unattainable due to physical and economic limitations. Then again, maybe even if we exhausted the global economy and every natural resource to build a planet-sized computer and simulated every minuscule part of a human brain, every cell, every neuron, and all the electro-chemical interactions, and it still wouldn't be conscious, maybe that is a far better result than if it could be conscious. What are we even doing? I ask you, what is the goal and benefit here? How would that be better than the unconscious AI we already have and the automatons we can already produce now, but haven't? For a time, computers will get smaller, faster, more powerful, but if we haven't already achieved Strong AI with the largest fastest computers available today, in the future if room-sized clusters can fit in a cell phone, how can we expect them to be conscious? And if the room sized computers of the future can be made conscious, how will that benefit anyone? > I'm not sure there is a third possibility? Well, imagine having two separate conscious brains, the one in your head and another somewhere else that is also physically a part of you. Maybe we'll alter our species to evolve these separate brains, I just don't know what the point would be, as it would not at all be like multiprocessing or parallel computing, but more like slavery.
  - jungturk 4y ago
    A sustainable process that can evolve healthy brains is the source of consciousness.
eru 4y ago
> Machines that learn, sure, but conscious machines, no way. So what? To have implications for the Fermi paradox, we merely need machines that can replicate throughout the galaxy. Whether they are conscious or not is irrelevant.
- Maursault 4y ago
  Since I responded to and argued against the OP's specific assertion, "Seems pretty clear that we're on a path to machine intelligence," your reintroduction of Fermi paradox, even if it is the subject of the article, still becomes OT in response to my comment, which has nothing to do with Fermi Paradox. In fact, I agree with you that it doesn't matter that machines will never be conscious, I'm just not going to get sucked into arguing about Fermi Paradox because I don't have any position on it. And I might as well argue, "so what? The pizza's here."
- akiselev 4y ago
  I doubt any civilizations capable of robotic interstellar travel would consider unleashing a Replicator-esque paper clip maximizer into the universe a worthwhile pursuit.
  - eru 4y ago
    Civilisations don't consider anything a worthwhile pursuit, because Civilisations don't consider anything as anything in the first place. Civilisations don't make choices or act. Individuals do. Already there's plenty of humans who do all kinds of weird things. In an interstellar future there will be orders of magnitudes more beings and thanks to light lag, no means of asserting overarching central authority.
TeMPOraL 4y ago
> Machines that learn, sure, but conscious machines, no way. If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns. I don't have a handy list of papers to support this, but I can recommend two novels by Peter Watts, Blindsight and Echopraxia, both of which deal with this idea (apologies for sort-of spoiler), and which both do come with references to academic publications backing this and other ideas used in the books.
- naasking 4y ago
  > If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns Considering we don't know what consciousness is or what it does functionally, any such argument is a fairy tale at best. It might contain some interesting insights or lessons, but don't mistake it for reality.
naasking 4y ago
> I'm not sure what you mean by "machine intelligence," but there is absolutely no indication that Strong AI can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible There is no compelling argument or evidence that machine intelligence is impossible. Physics tells us that machine intelligence is in fact very possible. All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). The only way to escape this conclusion is to assert that humans are special in some way that can't be described by physics. You can believe that, but don't mistake that as "evidence" or a scientific argument that machine intelligence is impossible. The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast.
- Maursault 4y ago
  > There is no compelling argument or evidence that machine intelligence is impossible. This is proof by assertion, but let's define what we're talking about first. I was referring to Strong AI. If you are also referring to Strong AI, then I can prove you wrong with not just one but two examples of compelling arguments against the possibility of Strong AI. Roger Penrose argues computers are unable to have intelligence because they are algorithmically deterministic systems. According to Penrose, rational processes of the mind are not completely algorithmic and therefore can not be duplicated by any sufficiently complex computer. Another compelling argument against the possibility of Strong AI is John Searle's Chinese Room thought experiment, and we know it is compelling for the Herculean amount of effort put into trying to prove it wrong for the last 40 years. > Physics tells us that machine intelligence is in fact very possible. This is more proof by assertion. You are asserting and not supporting your assertions in any way. > All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). It does not remotely follow from the Bekenstein bound that it is even humanly possible to perfectly describe a human, even if it sets an upper limit on the amount of information if it were possible, which it isn't, nor that perfectly describing a human will produce consciousness. A description is just that. A description of a book is not the book itself, a description of a human is not a human, and a description of consciousness is not consciousness, nor indeed could a description of consciousness itself be conscious. Further, the Bekenstein bound "implies that non-finite models such as Turing machines are not realizable as finite devices."[1] > The only way to escape this conclusion... More proof by assertion. You are asserting and not supporting your assertions, thus your argument is fallacious. Another way to escape your conclusion is to point out that you are misapplying the Bekenstein bound to falsely suggest that the limits it sets are or will ever be realized by humans, which they won't. We simply aren't capable and never will be capable of perfectly describing a human down to the quantum level. > The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast. More proof by assertion. Assertions are not proof, nor are they valid argument. [1] https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound
unsupp0rted 4y ago
Why would humans be conscious but machines not? If we replicated a human brain cell by cell wouldn’t that thing be conscious? Maybe we’d have to replicate a gut for it too, as a secondary brain, but then wouldn’t that thing be conscious? And if we started replacing parts of it with plastic, transistors and CPUs wouldn’t that thing still be conscious
- Maursault 4y ago
  > If we replicated a human brain cell by cell wouldn’t that thing be conscious? If we copied a human brain, assuming it were possible, what we'd be left with is two human brains. But I doubt either could be conscious, but even if they both were, they're still human brains, not computers. The mistake is confusing artificial intelligence with human intelligence, and the exercise would be like something out of a Mary Shelley novel and not computer science.
  - ben_w 4y ago
    A computer can simulate a physical process; if the intelligence in a human brain is physical in origin, a sufficiently powerful computer can simulate it. Such a simulation can reasonably be described as a "copy". The question then is "how powerful a computer", and nobody knows that, because even the lower bound estimates are too expensive to bother with at present.
    - Maursault 4y ago
      A simulation is not the real thing. We can simulate forces and interactions between simulated particles, but neither would have the actual properties of the real thing, only simulated properties. Simulated magnetism is not magnetic. It is just an appearance, an illusion. Consciousness only arises from actual, genuine healthy brain, not simulated brain. A simulated brain can no more be conscious than a video game.
      - unsupp0rted 4y ago
        A simulation of accounting data is accounting data. A simulation of two entities swapping data and creating a new mixed third entity is reproduction. It’s feasible that a simulation of a brain would be a brain.
ben_w 4y ago
By way of example: """ I'm not sure what you mean by "fleshy meatbag intelligence," but there is absolutely no indication that Strong Chemical Intelligence can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible. Fleshy meatbags that learn, sure, but conscious fleshy meatbags, no way. """ Does organic chemistry have a magical property that machines cannot have?
- lm28469 4y ago
  Well, we don't know, with out limited brains we very well might not be able to come up with machines capable of equalling or surpassing organic life, it doesn't have to be magical
  - ben_w 4y ago
    Perhaps, though that seems like a very different argument than what I replied to. I think (medium certainty) that because human minds were produced by evolution, which we can replicate trivially in simulations, that it is unlikely to be impossible for us to get there.
    - lm28469 4y ago
      What do you mean by "which we can replicate trivially in simulations" ?
      - ben_w 4y ago
        Implementing simulated evolution took me half an hour, while bored, 2 years after I graduated. :)
        simiones 4y ago
        Have any of your simulated evolution agents become as intelligent as, say, slime mould? If not, then I'm not sure the claim is meaningful (though, for what it's worth, I do agree with the gist of your argument).
poulpy123 4y ago
> there is absolutely no indication that Strong AI can be achieved l We have the proof that the I in AI can be achieved (us) and we have the proof that the A in AI can be achieved (our technology), so while there is no proof that AI can be achieved, I'm pretty sure there is nothing fundamental that prevent it
- lm28469 4y ago
  The fact that it could theoretically be achieved doesn't mean that _we_ can achieve it. For example the way we build computers might be super archaic and the equivalent of the flint stone, a flint stone is nice but if you need a fusion reactor no amount of flint stones will do the trick

Udo 4y ago

What's your objection to "conscious" machines? (Putting it in quotes because that's a loaded, unscientific term and everybody has their own definition)

Maursault 4y ago
My objection is it would be cruel and horrifying. Also, as of yet, we don't understand what mind is, and we can slice up the brain in a thousand ways and never find it. We may never know, and all we do know is that consciousness is a phenomenon arising from healthy brain. So how are we supposed to make a machine conscious without knowing how we ourselves are conscious? Dr. John Searle's objection is that software is not a mind, and goes on to explain why with his Chinese Room thought experiment.[1] [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- Udo 4y ago
  How do you know you're not interacting with a Chinese Room when you talk to another human being? Our own brains are an excellent proof of concept that machines can be made to think. We're all machines in a way, after all - unless you believe there is a special supernatural exception for human minds.
  - Maursault 4y ago
    Let's not again confuse natural intelligence with artificial intelligence. You do not have artificial consciousness, you have genuine consciousness. Further, the brain is not a machine. To explain why, I'm just going to quote this excellent piece from mindmatters: A machine is an artifact. It is a human-built assembly of materials that have no natural inclination to work in unison. Silicon and copper, plastic and glass have no natural propensity to compute. We make artifacts, for example computers, out of these materials to serve our purposes. But they are not the materials’ natural purposes. In the classical hylemorphic (Aristotelian) perspective, an artifact is an artificial composite of substances and thus it has an accidental, not substantial, form. Machines work as they do “accidentally”; that is, their function as machines is not inherent to the matter that comprises them. Rather their function is imposed on the disparate parts by human intelligence. Unlike a machine, the brain is an organ, a functional part of a living organism. It (along with the body) has a substantial form; its activity is natural to it. One might say that the brain's activity is intrinsic to it, not imposed on it extrinsically, as is the activity of a machine.[1] [1] https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-if-you-choose-to-see-it-that-way/ https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-i...
    - TeMPOraL 4y ago
      The brain absolutely is a machine. Just like everything biological in general: life is nanotech. We know that, because we can literally look through our microscopes and see how it ticks, at the atomic level. So we looked and saw that life is just molecular nanotech, but what we didn't see was any tags attached to carbon chains that would say, "this molecule's natural function is to think", or "that molecule's natural function is to be a part of an organ that pumps blood", etc. Because there is no such thing like a "natural function" in the sense used by your quote, the distinction between "natural" and "artificial" is itself artificial[0], and that whole quote represents a piece of magical thinking philosophy you'd think humanity abandoned at least a century ago. ---- [0] - https://abstrusegoose.com/215 https://abstrusegoose.com/215
      - Maursault 4y ago
        Words do have meanings, and "artificial" has a distinct meaning that is not the same meaning as that of "natural." Flavors natural and artificial are distinct from each other. There are natural functions. Pumping and circulating blood is a natural function of the heart. If you don't like "artificial," we can use "synthetic" or substitute "man-made." You have repeated assertions without any supporting argument. "The brain is absolutely a machine," is an assertion, but you have not remotely supported it. Proof by assertion[1] is fallacious argument. [1] https://en.wikipedia.org/wiki/Proof_by_assertion https://en.wikipedia.org/wiki/Proof_by_assertion
- eru 4y ago
  > My objection is it would be cruel and horrifying. That's an argument for why we _shouldn't_ build them. But not an argument for why we _can't_ build them. Do you see the difference? > So how are we supposed to make a machine conscious without knowing how we ourselves are conscious? Evolution didn't 'know' about conscious either. The pioneers of the Industrial Revolution had only the faintest ideas about thermodynamics, yet managed to build working steam engines. (And we improved them over time, as we learned more.) For the 'Chinese Room' see eg https://www.scottaaronson.com/democritus/lec4.html https://www.scottaaronson.com/democritus/lec4.html or just have a look at good old Wikipedia: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room Though I'm not sure why you brought up consciousness in the first place. It's irrelevant for the Fermi Paradox.
  - Maursault 4y ago
    > That's an argument for why we _shouldn't_ build them. But not an argument for why we _can't_ build them. Do you see the difference? This is a straw man fallacy, because I was asked "What's your objection to "conscious" machines?", and my objection is that it is cruel. > Evolution didn't 'know' about conscious either. The pioneers of the Industrial Revolution had only the faintest ideas about thermodynamics, yet managed to build working steam engines. (And we improved them over time, as we learned more.) This is also a fallacious argument. Just because we don't know how evolution created consciousness doesn't mean that we can use our ignorance of how consciousness evolved to create machine consciousness. If you're only using this as evidence that consciousness is possible, the fallacy is begging the question, petitio principii. Also, I believe there is here equivocation and conflation of artificial intelligence with natural intelligence. If we have the time, we can spend 600M years to evolve consciousness directly, but then it would not be artificial, it would not be machine intelligence, it would be natural or biological intelligence, and it would be a massive waste of time because it is already here and we are it. > For the 'Chinese Room' see eg https://www.scottaaronson.com/democritus/lec4.html https://www.scottaaronson.com/democritus/lec4.html or just have a look at good old Wikipedia: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room Do you really expect me to argue against your citations when you're too lazy even to summarize any particular counter-argument? It has been said that most of the work done in AI has been in trying to prove John Searle wrong, which suggests his argument is significant. A Turing Test does not prove consciousness nor even intelligence. All it proves is whether a human can be fooled. Arguments suggesting emergent properties between a human and a rule book are patently ridiculous, absurd on their face, because learning a book could not possibly ever make a book conscious. Where the Chinese Room thought experiment breaks down is there is a human involved, and most counter-arguments get tripped up here forgetting that it is metaphor and there is no human inside a computer where these types of counter-arguments would ever come true: no matter how much machine learning you submit your program to, there is no mind to begin with that could ever become aware. > Though I'm not sure why you brought up consciousness in the first place. It's irrelevant for the Fermi Paradox. In fact, I did not, the OP of the tread asserted: "Seems pretty clear that we're on a path to machine intelligence," and I responded to that specifically and argued against the possibility.
- nullc 4y ago
  > Dr. John Searle's objection is that software is not a mind, and goes on to explain why with his Chinese Room thought experiment. I always found that experiment to be extremely unconvincing, to put it mildly. Using that framework I can prove that you are not conscious. I construct a "english room". You are in the back without any contact to the outside world. A Chinese speaker in the front can pass messages to you from the outside world, in English by laborious copying. Clearly you aren't conscious because the Chinese speaker can't understand the English. Or, alternatively, no one would dispute a single neuron is not conscious and can't understand english. I pass signals into your neurons and through some electrochemical magic you reply in english, yet the neurons involved don't understand english therefor you aren't conscious. All Searle has done is picked out one component of a composite system, concluded that in isolation it has no clue whats going on, and from that concluded the system as a whole doesn't. We have a hard time clearly defining consciousness, and it's not clear if some AGI would be conscious (maybe it's just an emergent phenomena)-- it's also not clear if there would be any particular advantage in it being so. But in any case, I don't think the chinese room is a useful argument for anything.

jimmcslim 4y ago

> Machines that learn, sure, but conscious machines, no way. So what is the source of consciousness then? If it is something supernatural, like a soul, then yes reproducing consciousness in a machine is impossible. But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? I'm not sure there is a third possibility?

Maursault 4y ago
> So what is the source of consciousness then? Healthy brain is the source of consciousness. > But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? Though commonly referred to as such in metaphor, computers are not brains. Computers also have hard limitations that are very different from the limitations of biological brains. Maybe it is possible to accurately simulate a healthy brain but is unattainable due to physical and economic limitations. Then again, maybe even if we exhausted the global economy and every natural resource to build a planet-sized computer and simulated every minuscule part of a human brain, every cell, every neuron, and all the electro-chemical interactions, and it still wouldn't be conscious, maybe that is a far better result than if it could be conscious. What are we even doing? I ask you, what is the goal and benefit here? How would that be better than the unconscious AI we already have and the automatons we can already produce now, but haven't? For a time, computers will get smaller, faster, more powerful, but if we haven't already achieved Strong AI with the largest fastest computers available today, in the future if room-sized clusters can fit in a cell phone, how can we expect them to be conscious? And if the room sized computers of the future can be made conscious, how will that benefit anyone? > I'm not sure there is a third possibility? Well, imagine having two separate conscious brains, the one in your head and another somewhere else that is also physically a part of you. Maybe we'll alter our species to evolve these separate brains, I just don't know what the point would be, as it would not at all be like multiprocessing or parallel computing, but more like slavery.
- jungturk 4y ago
  A sustainable process that can evolve healthy brains is the source of consciousness.

eru 4y ago

> Machines that learn, sure, but conscious machines, no way. So what? To have implications for the Fermi paradox, we merely need machines that can replicate throughout the galaxy. Whether they are conscious or not is irrelevant.

Maursault 4y ago
Since I responded to and argued against the OP's specific assertion, "Seems pretty clear that we're on a path to machine intelligence," your reintroduction of Fermi paradox, even if it is the subject of the article, still becomes OT in response to my comment, which has nothing to do with Fermi Paradox. In fact, I agree with you that it doesn't matter that machines will never be conscious, I'm just not going to get sucked into arguing about Fermi Paradox because I don't have any position on it. And I might as well argue, "so what? The pizza's here."
akiselev 4y ago
I doubt any civilizations capable of robotic interstellar travel would consider unleashing a Replicator-esque paper clip maximizer into the universe a worthwhile pursuit.
- eru 4y ago
  Civilisations don't consider anything a worthwhile pursuit, because Civilisations don't consider anything as anything in the first place. Civilisations don't make choices or act. Individuals do. Already there's plenty of humans who do all kinds of weird things. In an interstellar future there will be orders of magnitudes more beings and thanks to light lag, no means of asserting overarching central authority.

TeMPOraL 4y ago

> Machines that learn, sure, but conscious machines, no way. If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns. I don't have a handy list of papers to support this, but I can recommend two novels by Peter Watts, Blindsight and Echopraxia, both of which deal with this idea (apologies for sort-of spoiler), and which both do come with references to academic publications backing this and other ideas used in the books.

naasking 4y ago
> If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns Considering we don't know what consciousness is or what it does functionally, any such argument is a fairy tale at best. It might contain some interesting insights or lessons, but don't mistake it for reality.

naasking 4y ago

> I'm not sure what you mean by "machine intelligence," but there is absolutely no indication that Strong AI can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible There is no compelling argument or evidence that machine intelligence is impossible. Physics tells us that machine intelligence is in fact very possible. All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). The only way to escape this conclusion is to assert that humans are special in some way that can't be described by physics. You can believe that, but don't mistake that as "evidence" or a scientific argument that machine intelligence is impossible. The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast.

Maursault 4y ago
> There is no compelling argument or evidence that machine intelligence is impossible. This is proof by assertion, but let's define what we're talking about first. I was referring to Strong AI. If you are also referring to Strong AI, then I can prove you wrong with not just one but two examples of compelling arguments against the possibility of Strong AI. Roger Penrose argues computers are unable to have intelligence because they are algorithmically deterministic systems. According to Penrose, rational processes of the mind are not completely algorithmic and therefore can not be duplicated by any sufficiently complex computer. Another compelling argument against the possibility of Strong AI is John Searle's Chinese Room thought experiment, and we know it is compelling for the Herculean amount of effort put into trying to prove it wrong for the last 40 years. > Physics tells us that machine intelligence is in fact very possible. This is more proof by assertion. You are asserting and not supporting your assertions in any way. > All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). It does not remotely follow from the Bekenstein bound that it is even humanly possible to perfectly describe a human, even if it sets an upper limit on the amount of information if it were possible, which it isn't, nor that perfectly describing a human will produce consciousness. A description is just that. A description of a book is not the book itself, a description of a human is not a human, and a description of consciousness is not consciousness, nor indeed could a description of consciousness itself be conscious. Further, the Bekenstein bound "implies that non-finite models such as Turing machines are not realizable as finite devices."[1] > The only way to escape this conclusion... More proof by assertion. You are asserting and not supporting your assertions, thus your argument is fallacious. Another way to escape your conclusion is to point out that you are misapplying the Bekenstein bound to falsely suggest that the limits it sets are or will ever be realized by humans, which they won't. We simply aren't capable and never will be capable of perfectly describing a human down to the quantum level. > The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast. More proof by assertion. Assertions are not proof, nor are they valid argument. [1] https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound

unsupp0rted 4y ago

Why would humans be conscious but machines not? If we replicated a human brain cell by cell wouldn’t that thing be conscious? Maybe we’d have to replicate a gut for it too, as a secondary brain, but then wouldn’t that thing be conscious? And if we started replacing parts of it with plastic, transistors and CPUs wouldn’t that thing still be conscious

Maursault 4y ago
> If we replicated a human brain cell by cell wouldn’t that thing be conscious? If we copied a human brain, assuming it were possible, what we'd be left with is two human brains. But I doubt either could be conscious, but even if they both were, they're still human brains, not computers. The mistake is confusing artificial intelligence with human intelligence, and the exercise would be like something out of a Mary Shelley novel and not computer science.
- ben_w 4y ago
  A computer can simulate a physical process; if the intelligence in a human brain is physical in origin, a sufficiently powerful computer can simulate it. Such a simulation can reasonably be described as a "copy". The question then is "how powerful a computer", and nobody knows that, because even the lower bound estimates are too expensive to bother with at present.
  - Maursault 4y ago
    A simulation is not the real thing. We can simulate forces and interactions between simulated particles, but neither would have the actual properties of the real thing, only simulated properties. Simulated magnetism is not magnetic. It is just an appearance, an illusion. Consciousness only arises from actual, genuine healthy brain, not simulated brain. A simulated brain can no more be conscious than a video game.
    - unsupp0rted 4y ago
      A simulation of accounting data is accounting data. A simulation of two entities swapping data and creating a new mixed third entity is reproduction. It’s feasible that a simulation of a brain would be a brain.

ben_w 4y ago

By way of example: """ I'm not sure what you mean by "fleshy meatbag intelligence," but there is absolutely no indication that Strong Chemical Intelligence can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible. Fleshy meatbags that learn, sure, but conscious fleshy meatbags, no way. """ Does organic chemistry have a magical property that machines cannot have?

lm28469 4y ago
Well, we don't know, with out limited brains we very well might not be able to come up with machines capable of equalling or surpassing organic life, it doesn't have to be magical
- ben_w 4y ago
  Perhaps, though that seems like a very different argument than what I replied to. I think (medium certainty) that because human minds were produced by evolution, which we can replicate trivially in simulations, that it is unlikely to be impossible for us to get there.
  - lm28469 4y ago
    What do you mean by "which we can replicate trivially in simulations" ?
    - ben_w 4y ago
      Implementing simulated evolution took me half an hour, while bored, 2 years after I graduated. :)
      - simiones 4y ago
        Have any of your simulated evolution agents become as intelligent as, say, slime mould? If not, then I'm not sure the claim is meaningful (though, for what it's worth, I do agree with the gist of your argument).

poulpy123 4y ago

> there is absolutely no indication that Strong AI can be achieved l We have the proof that the I in AI can be achieved (us) and we have the proof that the A in AI can be achieved (our technology), so while there is no proof that AI can be achieved, I'm pretty sure there is nothing fundamental that prevent it

lm28469 4y ago
The fact that it could theoretically be achieved doesn't mean that _we_ can achieve it. For example the way we build computers might be super archaic and the equivalent of the flint stone, a flint stone is nice but if you need a fusion reactor no amount of flint stones will do the trick