13 ms·

This seems like a bad assumption: > stars suitable for settlement should provide “environments nearly identical to that of the home planet.” Seems pretty clea

by roca 4y ago

This seems like a bad assumption: > stars suitable for settlement should provide “environments nearly identical to that of the home planet.” Seems pretty clear that we're on a path to machine intelligence, and that interstellar voyages will be undertaken by those intelligences, not meatbags. That humans like a class G sun is irrelevant to the prospering of those intelligences.

Maursault 4y ago
> Seems pretty clear that we're on a path to machine intelligence I'm not sure what you mean by "machine intelligence," but there is absolutely no indication that Strong AI can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible. Machines that learn, sure, but conscious machines, no way. Your droids, we don't serve their kind here. They'll have to wait outside. We don't want them here.
- Udo 4y ago
  What's your objection to "conscious" machines? (Putting it in quotes because that's a loaded, unscientific term and everybody has their own definition)
  - Maursault 4y ago
    My objection is it would be cruel and horrifying. Also, as of yet, we don't understand what mind is, and we can slice up the brain in a thousand ways and never find it. We may never know, and all we do know is that consciousness is a phenomenon arising from healthy brain. So how are we supposed to make a machine conscious without knowing how we ourselves are conscious? Dr. John Searle's objection is that software is not a mind, and goes on to explain why with his Chinese Room thought experiment.[1] [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
    - Udo 4y ago
      How do you know you're not interacting with a Chinese Room when you talk to another human being? Our own brains are an excellent proof of concept that machines can be made to think. We're all machines in a way, after all - unless you believe there is a special supernatural exception for human minds.
      - Maursault 4y ago
        Let's not again confuse natural intelligence with artificial intelligence. You do not have artificial consciousness, you have genuine consciousness. Further, the brain is not a machine. To explain why, I'm just going to quote this excellent piece from mindmatters: A machine is an artifact. It is a human-built assembly of materials that have no natural inclination to work in unison. Silicon and copper, plastic and glass have no natural propensity to compute. We make artifacts, for example computers, out of these materials to serve our purposes. But they are not the materials’ natural purposes. In the classical hylemorphic (Aristotelian) perspective, an artifact is an artificial composite of substances and thus it has an accidental, not substantial, form. Machines work as they do “accidentally”; that is, their function as machines is not inherent to the matter that comprises them. Rather their function is imposed on the disparate parts by human intelligence. Unlike a machine, the brain is an organ, a functional part of a living organism. It (along with the body) has a substantial form; its activity is natural to it. One might say that the brain's activity is intrinsic to it, not imposed on it extrinsically, as is the activity of a machine.[1] [1] https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-if-you-choose-to-see-it-that-way/ https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-i...
        TeMPOraL 4y ago
        The brain absolutely is a machine. Just like everything biological in general: life is nanotech. We know that, because we can literally look through our microscopes and see how it ticks, at the atomic level. So we looked and saw that life is just molecular nanotech, but what we didn't see was any tags attached to carbon chains that would say, "this molecule's natural function is to think", or "that molecule's natural function is to be a part of an organ that pumps blood", etc. Because there is no such thing like a "natural function" in the sense used by your quote, the distinction between "natural" and "artificial" is itself artificial[0], and that whole quote represents a piece of magical thinking philosophy you'd think humanity abandoned at least a century ago. ---- [0] - https://abstrusegoose.com/215 https://abstrusegoose.com/215
- jimmcslim 4y ago
  > Machines that learn, sure, but conscious machines, no way. So what is the source of consciousness then? If it is something supernatural, like a soul, then yes reproducing consciousness in a machine is impossible. But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? I'm not sure there is a third possibility?
  - Maursault 4y ago
    > So what is the source of consciousness then? Healthy brain is the source of consciousness. > But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? Though commonly referred to as such in metaphor, computers are not brains. Computers also have hard limitations that are very different from the limitations of biological brains. Maybe it is possible to accurately simulate a healthy brain but is unattainable due to physical and economic limitations. Then again, maybe even if we exhausted the global economy and every natural resource to build a planet-sized computer and simulated every minuscule part of a human brain, every cell, every neuron, and all the electro-chemical interactions, and it still wouldn't be conscious, maybe that is a far better result than if it could be conscious. What are we even doing? I ask you, what is the goal and benefit here? How would that be better than the unconscious AI we already have and the automatons we can already produce now, but haven't? For a time, computers will get smaller, faster, more powerful, but if we haven't already achieved Strong AI with the largest fastest computers available today, in the future if room-sized clusters can fit in a cell phone, how can we expect them to be conscious? And if the room sized computers of the future can be made conscious, how will that benefit anyone? > I'm not sure there is a third possibility? Well, imagine having two separate conscious brains, the one in your head and another somewhere else that is also physically a part of you. Maybe we'll alter our species to evolve these separate brains, I just don't know what the point would be, as it would not at all be like multiprocessing or parallel computing, but more like slavery.
    - jungturk 4y ago
      A sustainable process that can evolve healthy brains is the source of consciousness.
- eru 4y ago
  > Machines that learn, sure, but conscious machines, no way. So what? To have implications for the Fermi paradox, we merely need machines that can replicate throughout the galaxy. Whether they are conscious or not is irrelevant.
  - Maursault 4y ago
    Since I responded to and argued against the OP's specific assertion, "Seems pretty clear that we're on a path to machine intelligence," your reintroduction of Fermi paradox, even if it is the subject of the article, still becomes OT in response to my comment, which has nothing to do with Fermi Paradox. In fact, I agree with you that it doesn't matter that machines will never be conscious, I'm just not going to get sucked into arguing about Fermi Paradox because I don't have any position on it. And I might as well argue, "so what? The pizza's here."
  - akiselev 4y ago
    I doubt any civilizations capable of robotic interstellar travel would consider unleashing a Replicator-esque paper clip maximizer into the universe a worthwhile pursuit.
    - eru 4y ago
      Civilisations don't consider anything a worthwhile pursuit, because Civilisations don't consider anything as anything in the first place. Civilisations don't make choices or act. Individuals do. Already there's plenty of humans who do all kinds of weird things. In an interstellar future there will be orders of magnitudes more beings and thanks to light lag, no means of asserting overarching central authority.
- TeMPOraL 4y ago
  > Machines that learn, sure, but conscious machines, no way. If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns. I don't have a handy list of papers to support this, but I can recommend two novels by Peter Watts, Blindsight and Echopraxia, both of which deal with this idea (apologies for sort-of spoiler), and which both do come with references to academic publications backing this and other ideas used in the books.
  - naasking 4y ago
    > If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns Considering we don't know what consciousness is or what it does functionally, any such argument is a fairy tale at best. It might contain some interesting insights or lessons, but don't mistake it for reality.
- naasking 4y ago
  > I'm not sure what you mean by "machine intelligence," but there is absolutely no indication that Strong AI can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible There is no compelling argument or evidence that machine intelligence is impossible. Physics tells us that machine intelligence is in fact very possible. All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). The only way to escape this conclusion is to assert that humans are special in some way that can't be described by physics. You can believe that, but don't mistake that as "evidence" or a scientific argument that machine intelligence is impossible. The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast.
  - Maursault 4y ago
    > There is no compelling argument or evidence that machine intelligence is impossible. This is proof by assertion, but let's define what we're talking about first. I was referring to Strong AI. If you are also referring to Strong AI, then I can prove you wrong with not just one but two examples of compelling arguments against the possibility of Strong AI. Roger Penrose argues computers are unable to have intelligence because they are algorithmically deterministic systems. According to Penrose, rational processes of the mind are not completely algorithmic and therefore can not be duplicated by any sufficiently complex computer. Another compelling argument against the possibility of Strong AI is John Searle's Chinese Room thought experiment, and we know it is compelling for the Herculean amount of effort put into trying to prove it wrong for the last 40 years. > Physics tells us that machine intelligence is in fact very possible. This is more proof by assertion. You are asserting and not supporting your assertions in any way. > All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). It does not remotely follow from the Bekenstein bound that it is even humanly possible to perfectly describe a human, even if it sets an upper limit on the amount of information if it were possible, which it isn't, nor that perfectly describing a human will produce consciousness. A description is just that. A description of a book is not the book itself, a description of a human is not a human, and a description of consciousness is not consciousness, nor indeed could a description of consciousness itself be conscious. Further, the Bekenstein bound "implies that non-finite models such as Turing machines are not realizable as finite devices."[1] > The only way to escape this conclusion... More proof by assertion. You are asserting and not supporting your assertions, thus your argument is fallacious. Another way to escape your conclusion is to point out that you are misapplying the Bekenstein bound to falsely suggest that the limits it sets are or will ever be realized by humans, which they won't. We simply aren't capable and never will be capable of perfectly describing a human down to the quantum level. > The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast. More proof by assertion. Assertions are not proof, nor are they valid argument. [1] https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound
- unsupp0rted 4y ago
  Why would humans be conscious but machines not? If we replicated a human brain cell by cell wouldn’t that thing be conscious? Maybe we’d have to replicate a gut for it too, as a secondary brain, but then wouldn’t that thing be conscious? And if we started replacing parts of it with plastic, transistors and CPUs wouldn’t that thing still be conscious
  - Maursault 4y ago
    > If we replicated a human brain cell by cell wouldn’t that thing be conscious? If we copied a human brain, assuming it were possible, what we'd be left with is two human brains. But I doubt either could be conscious, but even if they both were, they're still human brains, not computers. The mistake is confusing artificial intelligence with human intelligence, and the exercise would be like something out of a Mary Shelley novel and not computer science.
    - ben_w 4y ago
      A computer can simulate a physical process; if the intelligence in a human brain is physical in origin, a sufficiently powerful computer can simulate it. Such a simulation can reasonably be described as a "copy". The question then is "how powerful a computer", and nobody knows that, because even the lower bound estimates are too expensive to bother with at present.
      - Maursault 4y ago
        A simulation is not the real thing. We can simulate forces and interactions between simulated particles, but neither would have the actual properties of the real thing, only simulated properties. Simulated magnetism is not magnetic. It is just an appearance, an illusion. Consciousness only arises from actual, genuine healthy brain, not simulated brain. A simulated brain can no more be conscious than a video game.
        unsupp0rted 4y ago
        A simulation of accounting data is accounting data. A simulation of two entities swapping data and creating a new mixed third entity is reproduction. It’s feasible that a simulation of a brain would be a brain.
- ben_w 4y ago
  By way of example: """ I'm not sure what you mean by "fleshy meatbag intelligence," but there is absolutely no indication that Strong Chemical Intelligence can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible. Fleshy meatbags that learn, sure, but conscious fleshy meatbags, no way. """ Does organic chemistry have a magical property that machines cannot have?
  - lm28469 4y ago
    Well, we don't know, with out limited brains we very well might not be able to come up with machines capable of equalling or surpassing organic life, it doesn't have to be magical
    - ben_w 4y ago
      Perhaps, though that seems like a very different argument than what I replied to. I think (medium certainty) that because human minds were produced by evolution, which we can replicate trivially in simulations, that it is unlikely to be impossible for us to get there.
      - lm28469 4y ago
        What do you mean by "which we can replicate trivially in simulations" ?
        ben_w 4y ago
        Implementing simulated evolution took me half an hour, while bored, 2 years after I graduated. :)
        simiones 4y ago
        Have any of your simulated evolution agents become as intelligent as, say, slime mould? If not, then I'm not sure the claim is meaningful (though, for what it's worth, I do agree with the gist of your argument).
- poulpy123 4y ago
  > there is absolutely no indication that Strong AI can be achieved l We have the proof that the I in AI can be achieved (us) and we have the proof that the A in AI can be achieved (our technology), so while there is no proof that AI can be achieved, I'm pretty sure there is nothing fundamental that prevent it
  - lm28469 4y ago
    The fact that it could theoretically be achieved doesn't mean that _we_ can achieve it. For example the way we build computers might be super archaic and the equivalent of the flint stone, a flint stone is nice but if you need a fusion reactor no amount of flint stones will do the trick
Udo 4y ago
> That humans like a class G sun is irrelevant to the prospering of those intelligences. It's also worth noting that the types of star or planet available don't matter much to meatbags either once they have learned to leave their homeworld. Orbital or even deep space habitats are much more efficient and easier to construct than livable planets. As long as there is a reasonable energy source nearby, you can have any environment you like in a spinning cylinder. > and that interstellar voyages will be undertaken by those intelligences Probably, although I would much prefer if we humans became hybrids of some sort (maybe uploaded and virtualized, or extensively augmented) instead of just falling by the wayside. It would be sad if our only job in cosmic history turns out to be giving birth to the real main characters of the story.
- eru 4y ago
  > Probably, although I would much prefer if we humans became hybrids of some sort (maybe uploaded and virtualized, or extensively augmented) instead of just falling by the wayside. Well, would you consider yourself to be a heavily augmented fish? (Or heavily augmented amoeba?) If yes, you can probably re-interpret our successors in that light, too.
  - Udo 4y ago
    > Well, would you consider yourself to be a heavily augmented fish? Among other things, yes. There are enormous amounts of biochemical machinery we are still running today originating from the dawn of life. Although I believe intentionally designed augmentations would be (and in a way: already are) a complete game changer, in at least as much as the advent of higher reasoning and shared culture was the game changer that gave birth to us. So there may well be an insurmountable wall of separation between us and our successors. From my admittedly biased perspective I think the most important aspects worth porting over to a new species are not even necessarily biological, but cultural and emotional. I personally think there is a potential for experiential richness that I would love to see explored in a human-like but highly advanced lifeform. To be clear, I wouldn't object to personally being that lifeform, either, but alas I was born too early.
- Ekaros 4y ago
jimmytucson 4y ago
Not directly related to the article but since you brought it up, why would machine intelligences want to prosper and propagate throughout the galaxy? Biological lifeforms would to ensure the survival of their DNA. What about computer programs?
- Udo 4y ago
  > Biological lifeforms would to ensure the survival of their DNA. Humans are already a hybrid organism that works to ensure survival not only of genes but of culture and knowledge. It's not difficult to imagine a machine civilization would do the same: not only preserve their construction patterns but also their intellectual achievements. However, none of that is strictly necessary - because evolutionary pressure still applies to machine species in the same way as it applies to us. Machines that stay at home and don't propagate, well, won't propagate. Eventually someone comes along who does propagate and the stagnant species will fade away. There isn't really any difference in mechanism between what we consider biological lifeforms compared to machine ones. Edit @jimmytucson: HN doesn't let me respond directly, so I'm clarifying my answer here: > And I can’t see any reason why it would, tbh It doesn't technically need a reason. The fact that those machines who do replicate eventually dominate the landscape is enough, on its own. If 100 billion machine civilizations choose not to procreate but one does - that's the one that eventually seeds the ecosystem.
  - eru 4y ago
    Mostly agreed! > There isn't really any difference in mechanism between what we consider biological lifeforms compared to machine ones. There is one important difference: we can make machines with vastly better error correction for their replication. That has large impacts on any 'machine evolution'. Your argument still applies. But there might not be any noticeable change in the machines long after the stars burn out. Especially if initially they are engineered to have a long generation time. And if you take into account that most mutations are bad for fitness.
    - Udo 4y ago
      > we can make machines with vastly better error correction for their replication Absolutely, I think we agree on that. It's fun to think about whether machines with some intentionally elevated drift rate may occasionally outperform locked-down ones, but I doubt it would be worth it since I don't see a random walk algorithm like evolution beating intentional design strategies implemented by extreme intellects. (Especially since they'd be able to simulate designs beforehand and could easily consider vast numbers of variations in advance...)
jimmcslim 4y ago
For anyone unfamiliar with Charlie Stross' "Saturn's Children" and "Neptune's Brood", it explores a far future where humanity is more or less extinct but the galaxy is being populated by (conveniently similarly anthropological) AI androids.
- eru 4y ago
  The novels you mention also have AIs housed in bodies that don't resemble humans at all. (If memory serves right, the miners in the outer system have rather different bodies.)
- sneak 4y ago
  Are your children still human? The things being created and reproducing will still be humanity, just not meat-based humanity.
  - HideousKojima 4y ago
    Children are human, a chair I make out of wood isn't.
    - sneak 4y ago
      A chair isn't sentient. A sentient being that you make out of anything is definitionally your child.
JohnJamesRambo 4y ago
I’m not so sure about that. It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach with regard to not getting stuck on stuff in the real world, constantly having energy, able to repair itself in the physical world. I think people hand wave these things away and they are very huge hurdles. Even something as simple as a car can’t run a decade autonomously. We expect it to explore the stars, make robots that can remove circuit boards, troubleshoot, etc.? Nothing around me keeps working without human interaction for even very small timescales.
- knodi123 4y ago
  > It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach Ah, but the trick is, as soon as you can get a machine intelligence smart enough to work on programming tasks, you can get it to help you design a better one. And so on, etc. The first AGI is (some) amount of time away. A more sophisticated AGI is coming not long after that, and exponentially faster and so on.
  - asdff 4y ago
    Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated? For example with our own intelligence, we aren't getting smarter with each generation. We have less brain matter than humans who lived millions of years ago on average, because there isn't as strong of a selective pressure present these days to see those with a lot of brain matter out reproduce those who get by in life with not as much of it. A very 'intelligent' AGI might realize the most optimal way to seed the planet with itself is actually to have a swarm of very unintelligent AI that is able to run on some very non resource intensive hardware, and seed it widely like how an "unintelligent" insect is able to lay millions of eggs and infest an area beyond human control, and the more intelligent wolf is easily hunted down.
    - pencilguin 4y ago
      We do not, in fact, have less brain than hominins millions of years ago. We do have less than our immediate ancestors 100,000 years ago. Your very intelligent AGI would most likely have no interest in seeding anything on Earth. It might reasonably be curious about our ecosystems, but disturbing those would reduce that interest.
cletus 4y ago
> ... interstellar voyages will be undertaken by those intelligences, not meatbags. The energy requirements for interstellar travel are so vast that the life support requirements for thousands of meatbags for possibly thousands of years really are just a rounding error.
- cwillu 4y ago
  It's only a rounding error because rounding 10¹⁸ to 10²⁰ is a thing you do in astrophysics.
  - ben_w 4y ago
    Kinetic energy of 100kg at 0.1c is ≈ 4.5e16 J Minimum possible life support for a human ≈ human basic metabolic rate ≈ 73 W Divide energy by power to get time; that's 20 million years, which at that speed would get you to Andromeda before the lowest possible energy cost of life support reached the lowest possible energy cost of the velocity.
asdff 4y ago
Seriously. If we want to seed the galaxy with earth based life, we are a lot better off launching probes full of tardigrades, yeasts, and bacteria.
mapt 4y ago
You get three tiers propagating at very different speeds: The machine intelligence - spread solely over the airwaves to receptive civilizations, which then get taken over to promulgate further spread. Laser launch telescopes have great bandwidth and transmit at approximately the speed of light, but require some selectivity. The intelligent machine - Robotic Von Neumann probes, probably of the star-wisp design. Can colonize barren rocks, and be sent out in large batches at low speed. The intelligent race - The fleshy bits. Not even very interested in barren rocks compared to planets. So logistically difficult that the supporting infrastructure limits them to tiny fractions of the speed of light at the cost of large fractions of civilizational output.
rrwo 4y ago
You are assuming Earth-like "meatbags". A suitably advanced civilization might be able to genetically re-engineer their bodies with live longer, and in more exotic environments. They may also go the cyborg route and have mechanical appendages to their bodies, rather than replace their bodies entirely. They may also be able to do things like put their bodies in a state of hibernation during interstellar travel.

by roca 4y ago

Maursault 4y ago

> Seems pretty clear that we're on a path to machine intelligence I'm not sure what you mean by "machine intelligence," but there is absolutely no indication that Strong AI can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible. Machines that learn, sure, but conscious machines, no way. Your droids, we don't serve their kind here. They'll have to wait outside. We don't want them here.

Udo 4y ago
What's your objection to "conscious" machines? (Putting it in quotes because that's a loaded, unscientific term and everybody has their own definition)
- Maursault 4y ago
  My objection is it would be cruel and horrifying. Also, as of yet, we don't understand what mind is, and we can slice up the brain in a thousand ways and never find it. We may never know, and all we do know is that consciousness is a phenomenon arising from healthy brain. So how are we supposed to make a machine conscious without knowing how we ourselves are conscious? Dr. John Searle's objection is that software is not a mind, and goes on to explain why with his Chinese Room thought experiment.[1] [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
  - Udo 4y ago
    How do you know you're not interacting with a Chinese Room when you talk to another human being? Our own brains are an excellent proof of concept that machines can be made to think. We're all machines in a way, after all - unless you believe there is a special supernatural exception for human minds.
    - Maursault 4y ago
      Let's not again confuse natural intelligence with artificial intelligence. You do not have artificial consciousness, you have genuine consciousness. Further, the brain is not a machine. To explain why, I'm just going to quote this excellent piece from mindmatters: A machine is an artifact. It is a human-built assembly of materials that have no natural inclination to work in unison. Silicon and copper, plastic and glass have no natural propensity to compute. We make artifacts, for example computers, out of these materials to serve our purposes. But they are not the materials’ natural purposes. In the classical hylemorphic (Aristotelian) perspective, an artifact is an artificial composite of substances and thus it has an accidental, not substantial, form. Machines work as they do “accidentally”; that is, their function as machines is not inherent to the matter that comprises them. Rather their function is imposed on the disparate parts by human intelligence. Unlike a machine, the brain is an organ, a functional part of a living organism. It (along with the body) has a substantial form; its activity is natural to it. One might say that the brain's activity is intrinsic to it, not imposed on it extrinsically, as is the activity of a machine.[1] [1] https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-if-you-choose-to-see-it-that-way/ https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-i...
      - TeMPOraL 4y ago
        The brain absolutely is a machine. Just like everything biological in general: life is nanotech. We know that, because we can literally look through our microscopes and see how it ticks, at the atomic level. So we looked and saw that life is just molecular nanotech, but what we didn't see was any tags attached to carbon chains that would say, "this molecule's natural function is to think", or "that molecule's natural function is to be a part of an organ that pumps blood", etc. Because there is no such thing like a "natural function" in the sense used by your quote, the distinction between "natural" and "artificial" is itself artificial[0], and that whole quote represents a piece of magical thinking philosophy you'd think humanity abandoned at least a century ago. ---- [0] - https://abstrusegoose.com/215 https://abstrusegoose.com/215
jimmcslim 4y ago
> Machines that learn, sure, but conscious machines, no way. So what is the source of consciousness then? If it is something supernatural, like a soul, then yes reproducing consciousness in a machine is impossible. But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? I'm not sure there is a third possibility?
- Maursault 4y ago
  > So what is the source of consciousness then? Healthy brain is the source of consciousness. > But if it is ultimately just a biological process, no matter how complicated, then it could be simulated given true understanding and a fast enough/big enough computer? Though commonly referred to as such in metaphor, computers are not brains. Computers also have hard limitations that are very different from the limitations of biological brains. Maybe it is possible to accurately simulate a healthy brain but is unattainable due to physical and economic limitations. Then again, maybe even if we exhausted the global economy and every natural resource to build a planet-sized computer and simulated every minuscule part of a human brain, every cell, every neuron, and all the electro-chemical interactions, and it still wouldn't be conscious, maybe that is a far better result than if it could be conscious. What are we even doing? I ask you, what is the goal and benefit here? How would that be better than the unconscious AI we already have and the automatons we can already produce now, but haven't? For a time, computers will get smaller, faster, more powerful, but if we haven't already achieved Strong AI with the largest fastest computers available today, in the future if room-sized clusters can fit in a cell phone, how can we expect them to be conscious? And if the room sized computers of the future can be made conscious, how will that benefit anyone? > I'm not sure there is a third possibility? Well, imagine having two separate conscious brains, the one in your head and another somewhere else that is also physically a part of you. Maybe we'll alter our species to evolve these separate brains, I just don't know what the point would be, as it would not at all be like multiprocessing or parallel computing, but more like slavery.
  - jungturk 4y ago
    A sustainable process that can evolve healthy brains is the source of consciousness.
eru 4y ago
> Machines that learn, sure, but conscious machines, no way. So what? To have implications for the Fermi paradox, we merely need machines that can replicate throughout the galaxy. Whether they are conscious or not is irrelevant.
- Maursault 4y ago
  Since I responded to and argued against the OP's specific assertion, "Seems pretty clear that we're on a path to machine intelligence," your reintroduction of Fermi paradox, even if it is the subject of the article, still becomes OT in response to my comment, which has nothing to do with Fermi Paradox. In fact, I agree with you that it doesn't matter that machines will never be conscious, I'm just not going to get sucked into arguing about Fermi Paradox because I don't have any position on it. And I might as well argue, "so what? The pizza's here."
- akiselev 4y ago
  I doubt any civilizations capable of robotic interstellar travel would consider unleashing a Replicator-esque paper clip maximizer into the universe a worthwhile pursuit.
  - eru 4y ago
    Civilisations don't consider anything a worthwhile pursuit, because Civilisations don't consider anything as anything in the first place. Civilisations don't make choices or act. Individuals do. Already there's plenty of humans who do all kinds of weird things. In an interstellar future there will be orders of magnitudes more beings and thanks to light lag, no means of asserting overarching central authority.
TeMPOraL 4y ago
> Machines that learn, sure, but conscious machines, no way. If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns. I don't have a handy list of papers to support this, but I can recommend two novels by Peter Watts, Blindsight and Echopraxia, both of which deal with this idea (apologies for sort-of spoiler), and which both do come with references to academic publications backing this and other ideas used in the books.
- naasking 4y ago
  > If we're trading compelling arguments, then there is one for consciousness not only being unnecessary, but also a drawback that evolution would happily optimize away given time, and basically not worth the energy it burns Considering we don't know what consciousness is or what it does functionally, any such argument is a fairy tale at best. It might contain some interesting insights or lessons, but don't mistake it for reality.
naasking 4y ago
> I'm not sure what you mean by "machine intelligence," but there is absolutely no indication that Strong AI can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible There is no compelling argument or evidence that machine intelligence is impossible. Physics tells us that machine intelligence is in fact very possible. All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). The only way to escape this conclusion is to assert that humans are special in some way that can't be described by physics. You can believe that, but don't mistake that as "evidence" or a scientific argument that machine intelligence is impossible. The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast.
- Maursault 4y ago
  > There is no compelling argument or evidence that machine intelligence is impossible. This is proof by assertion, but let's define what we're talking about first. I was referring to Strong AI. If you are also referring to Strong AI, then I can prove you wrong with not just one but two examples of compelling arguments against the possibility of Strong AI. Roger Penrose argues computers are unable to have intelligence because they are algorithmically deterministic systems. According to Penrose, rational processes of the mind are not completely algorithmic and therefore can not be duplicated by any sufficiently complex computer. Another compelling argument against the possibility of Strong AI is John Searle's Chinese Room thought experiment, and we know it is compelling for the Herculean amount of effort put into trying to prove it wrong for the last 40 years. > Physics tells us that machine intelligence is in fact very possible. This is more proof by assertion. You are asserting and not supporting your assertions in any way. > All finite volumes contain finite information, per the Bekenstein bound. Therefore a human being can be fully described by finite information. Therefore a human being can be fully simulated by a Turing machine (technically just a finite state automaton). It does not remotely follow from the Bekenstein bound that it is even humanly possible to perfectly describe a human, even if it sets an upper limit on the amount of information if it were possible, which it isn't, nor that perfectly describing a human will produce consciousness. A description is just that. A description of a book is not the book itself, a description of a human is not a human, and a description of consciousness is not consciousness, nor indeed could a description of consciousness itself be conscious. Further, the Bekenstein bound "implies that non-finite models such as Turing machines are not realizable as finite devices."[1] > The only way to escape this conclusion... More proof by assertion. You are asserting and not supporting your assertions, thus your argument is fallacious. Another way to escape your conclusion is to point out that you are misapplying the Bekenstein bound to falsely suggest that the limits it sets are or will ever be realized by humans, which they won't. We simply aren't capable and never will be capable of perfectly describing a human down to the quantum level. > The current state of the art machine learning systems are still a few orders of magnitude away from the synaptic complexity of a human brains. A lot can happen over a few orders of magnitude, and we're closing that gap fast. More proof by assertion. Assertions are not proof, nor are they valid argument. [1] https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound
unsupp0rted 4y ago
Why would humans be conscious but machines not? If we replicated a human brain cell by cell wouldn’t that thing be conscious? Maybe we’d have to replicate a gut for it too, as a secondary brain, but then wouldn’t that thing be conscious? And if we started replacing parts of it with plastic, transistors and CPUs wouldn’t that thing still be conscious
- Maursault 4y ago
  > If we replicated a human brain cell by cell wouldn’t that thing be conscious? If we copied a human brain, assuming it were possible, what we'd be left with is two human brains. But I doubt either could be conscious, but even if they both were, they're still human brains, not computers. The mistake is confusing artificial intelligence with human intelligence, and the exercise would be like something out of a Mary Shelley novel and not computer science.
  - ben_w 4y ago
    A computer can simulate a physical process; if the intelligence in a human brain is physical in origin, a sufficiently powerful computer can simulate it. Such a simulation can reasonably be described as a "copy". The question then is "how powerful a computer", and nobody knows that, because even the lower bound estimates are too expensive to bother with at present.
    - Maursault 4y ago
      A simulation is not the real thing. We can simulate forces and interactions between simulated particles, but neither would have the actual properties of the real thing, only simulated properties. Simulated magnetism is not magnetic. It is just an appearance, an illusion. Consciousness only arises from actual, genuine healthy brain, not simulated brain. A simulated brain can no more be conscious than a video game.
      - unsupp0rted 4y ago
        A simulation of accounting data is accounting data. A simulation of two entities swapping data and creating a new mixed third entity is reproduction. It’s feasible that a simulation of a brain would be a brain.
ben_w 4y ago
By way of example: """ I'm not sure what you mean by "fleshy meatbag intelligence," but there is absolutely no indication that Strong Chemical Intelligence can be achieved let alone "pretty clear we're on a path," and compelling argument that it isn't even possible. Fleshy meatbags that learn, sure, but conscious fleshy meatbags, no way. """ Does organic chemistry have a magical property that machines cannot have?
- lm28469 4y ago
  Well, we don't know, with out limited brains we very well might not be able to come up with machines capable of equalling or surpassing organic life, it doesn't have to be magical
  - ben_w 4y ago
    Perhaps, though that seems like a very different argument than what I replied to. I think (medium certainty) that because human minds were produced by evolution, which we can replicate trivially in simulations, that it is unlikely to be impossible for us to get there.
    - lm28469 4y ago
      What do you mean by "which we can replicate trivially in simulations" ?
      - ben_w 4y ago
        Implementing simulated evolution took me half an hour, while bored, 2 years after I graduated. :)
        simiones 4y ago
        Have any of your simulated evolution agents become as intelligent as, say, slime mould? If not, then I'm not sure the claim is meaningful (though, for what it's worth, I do agree with the gist of your argument).
poulpy123 4y ago
> there is absolutely no indication that Strong AI can be achieved l We have the proof that the I in AI can be achieved (us) and we have the proof that the A in AI can be achieved (our technology), so while there is no proof that AI can be achieved, I'm pretty sure there is nothing fundamental that prevent it
- lm28469 4y ago
  The fact that it could theoretically be achieved doesn't mean that _we_ can achieve it. For example the way we build computers might be super archaic and the equivalent of the flint stone, a flint stone is nice but if you need a fusion reactor no amount of flint stones will do the trick

Udo 4y ago

> That humans like a class G sun is irrelevant to the prospering of those intelligences. It's also worth noting that the types of star or planet available don't matter much to meatbags either once they have learned to leave their homeworld. Orbital or even deep space habitats are much more efficient and easier to construct than livable planets. As long as there is a reasonable energy source nearby, you can have any environment you like in a spinning cylinder. > and that interstellar voyages will be undertaken by those intelligences Probably, although I would much prefer if we humans became hybrids of some sort (maybe uploaded and virtualized, or extensively augmented) instead of just falling by the wayside. It would be sad if our only job in cosmic history turns out to be giving birth to the real main characters of the story.

eru 4y ago
> Probably, although I would much prefer if we humans became hybrids of some sort (maybe uploaded and virtualized, or extensively augmented) instead of just falling by the wayside. Well, would you consider yourself to be a heavily augmented fish? (Or heavily augmented amoeba?) If yes, you can probably re-interpret our successors in that light, too.
- Udo 4y ago
  > Well, would you consider yourself to be a heavily augmented fish? Among other things, yes. There are enormous amounts of biochemical machinery we are still running today originating from the dawn of life. Although I believe intentionally designed augmentations would be (and in a way: already are) a complete game changer, in at least as much as the advent of higher reasoning and shared culture was the game changer that gave birth to us. So there may well be an insurmountable wall of separation between us and our successors. From my admittedly biased perspective I think the most important aspects worth porting over to a new species are not even necessarily biological, but cultural and emotional. I personally think there is a potential for experiential richness that I would love to see explored in a human-like but highly advanced lifeform. To be clear, I wouldn't object to personally being that lifeform, either, but alas I was born too early.
Ekaros 4y ago

jimmytucson 4y ago

Not directly related to the article but since you brought it up, why would machine intelligences want to prosper and propagate throughout the galaxy? Biological lifeforms would to ensure the survival of their DNA. What about computer programs?

Udo 4y ago
> Biological lifeforms would to ensure the survival of their DNA. Humans are already a hybrid organism that works to ensure survival not only of genes but of culture and knowledge. It's not difficult to imagine a machine civilization would do the same: not only preserve their construction patterns but also their intellectual achievements. However, none of that is strictly necessary - because evolutionary pressure still applies to machine species in the same way as it applies to us. Machines that stay at home and don't propagate, well, won't propagate. Eventually someone comes along who does propagate and the stagnant species will fade away. There isn't really any difference in mechanism between what we consider biological lifeforms compared to machine ones. Edit @jimmytucson: HN doesn't let me respond directly, so I'm clarifying my answer here: > And I can’t see any reason why it would, tbh It doesn't technically need a reason. The fact that those machines who do replicate eventually dominate the landscape is enough, on its own. If 100 billion machine civilizations choose not to procreate but one does - that's the one that eventually seeds the ecosystem.
- eru 4y ago
  Mostly agreed! > There isn't really any difference in mechanism between what we consider biological lifeforms compared to machine ones. There is one important difference: we can make machines with vastly better error correction for their replication. That has large impacts on any 'machine evolution'. Your argument still applies. But there might not be any noticeable change in the machines long after the stars burn out. Especially if initially they are engineered to have a long generation time. And if you take into account that most mutations are bad for fitness.
  - Udo 4y ago
    > we can make machines with vastly better error correction for their replication Absolutely, I think we agree on that. It's fun to think about whether machines with some intentionally elevated drift rate may occasionally outperform locked-down ones, but I doubt it would be worth it since I don't see a random walk algorithm like evolution beating intentional design strategies implemented by extreme intellects. (Especially since they'd be able to simulate designs beforehand and could easily consider vast numbers of variations in advance...)

jimmcslim 4y ago

For anyone unfamiliar with Charlie Stross' "Saturn's Children" and "Neptune's Brood", it explores a far future where humanity is more or less extinct but the galaxy is being populated by (conveniently similarly anthropological) AI androids.

eru 4y ago
The novels you mention also have AIs housed in bodies that don't resemble humans at all. (If memory serves right, the miners in the outer system have rather different bodies.)
sneak 4y ago
Are your children still human? The things being created and reproducing will still be humanity, just not meat-based humanity.
- HideousKojima 4y ago
  Children are human, a chair I make out of wood isn't.
  - sneak 4y ago
    A chair isn't sentient. A sentient being that you make out of anything is definitionally your child.

JohnJamesRambo 4y ago

I’m not so sure about that. It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach with regard to not getting stuck on stuff in the real world, constantly having energy, able to repair itself in the physical world. I think people hand wave these things away and they are very huge hurdles. Even something as simple as a car can’t run a decade autonomously. We expect it to explore the stars, make robots that can remove circuit boards, troubleshoot, etc.? Nothing around me keeps working without human interaction for even very small timescales.

knodi123 4y ago
> It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach Ah, but the trick is, as soon as you can get a machine intelligence smart enough to work on programming tasks, you can get it to help you design a better one. And so on, etc. The first AGI is (some) amount of time away. A more sophisticated AGI is coming not long after that, and exponentially faster and so on.
- asdff 4y ago
  Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated? For example with our own intelligence, we aren't getting smarter with each generation. We have less brain matter than humans who lived millions of years ago on average, because there isn't as strong of a selective pressure present these days to see those with a lot of brain matter out reproduce those who get by in life with not as much of it. A very 'intelligent' AGI might realize the most optimal way to seed the planet with itself is actually to have a swarm of very unintelligent AI that is able to run on some very non resource intensive hardware, and seed it widely like how an "unintelligent" insect is able to lay millions of eggs and infest an area beyond human control, and the more intelligent wolf is easily hunted down.
  - pencilguin 4y ago
    We do not, in fact, have less brain than hominins millions of years ago. We do have less than our immediate ancestors 100,000 years ago. Your very intelligent AGI would most likely have no interest in seeding anything on Earth. It might reasonably be curious about our ecosystems, but disturbing those would reduce that interest.

cletus 4y ago

> ... interstellar voyages will be undertaken by those intelligences, not meatbags. The energy requirements for interstellar travel are so vast that the life support requirements for thousands of meatbags for possibly thousands of years really are just a rounding error.

cwillu 4y ago
It's only a rounding error because rounding 10¹⁸ to 10²⁰ is a thing you do in astrophysics.
- ben_w 4y ago
  Kinetic energy of 100kg at 0.1c is ≈ 4.5e16 J Minimum possible life support for a human ≈ human basic metabolic rate ≈ 73 W Divide energy by power to get time; that's 20 million years, which at that speed would get you to Andromeda before the lowest possible energy cost of life support reached the lowest possible energy cost of the velocity.

asdff 4y ago

Seriously. If we want to seed the galaxy with earth based life, we are a lot better off launching probes full of tardigrades, yeasts, and bacteria.

mapt 4y ago

You get three tiers propagating at very different speeds: The machine intelligence - spread solely over the airwaves to receptive civilizations, which then get taken over to promulgate further spread. Laser launch telescopes have great bandwidth and transmit at approximately the speed of light, but require some selectivity. The intelligent machine - Robotic Von Neumann probes, probably of the star-wisp design. Can colonize barren rocks, and be sent out in large batches at low speed. The intelligent race - The fleshy bits. Not even very interested in barren rocks compared to planets. So logistically difficult that the supporting infrastructure limits them to tiny fractions of the speed of light at the cost of large fractions of civilizational output.

rrwo 4y ago

You are assuming Earth-like "meatbags". A suitably advanced civilization might be able to genetically re-engineer their bodies with live longer, and in more exotic environments. They may also go the cyborg route and have mechanical appendages to their bodies, rather than replace their bodies entirely. They may also be able to do things like put their bodies in a state of hibernation during interstellar travel.