6 ms·

I don't fear AI for the same reason I don't fear nanotechnology - nature tried to take over the world first. The grey goo scenario, where nanobots try and turn

by tomlock 12y ago

I don't fear AI for the same reason I don't fear nanotechnology - nature tried to take over the world first. The grey goo scenario, where nanobots try and turn the world into more nanobots, resulting in the consumption of all humans and buildings and bridges seems scary until you realize that this is exactly the goal of all bacteria. The idea of an entity that thirsts for power and sees competing ideologies as a threat doesn't seem that unfamiliar if you look at humans through history. Additionally, AI-fearers have a grand theory that we'll be able to create a machine that improves itself better than evolution has tried to. Datacenters require maintenance, and so the machine will need to entirely organize that before it can self-sustain. This may create a resource load that diminishes the ability to take over the world, much like my need for food diminishes my ability to do so. It feels like AI today has a lot less redundancy to overcome the edge-failures which cause a permanent shutdown, and which make the human brain seem a little slow when computing floating-point division. The idea that an exponentially self-improving being will arise seems unlikely when nature has been trying to do that for eons. The idea that we'll be the ones to find the secret sauce seems unlikely, but maybe its not so surprising that people who made their millions, and saw a parabolic rise of their own power thanks to technology, see it as a barely-constrained threat.

Turing_Machine 12y ago
"you realize that this is exactly the goal of all bacteria." Bacteria evolved. Design can do much, much better. Natural photosynthesis is 11% efficient, max. Solar cells are already up to 44% in the lab. An efficient plant would be black, not green.
- tomlock 12y ago
  And leaves have a lot of self-repair features that solar cells do not. The design is good for efficiency, but is it good for creating a self-sustaining entity?
  - Turing_Machine 12y ago
    Replication is enough. Self-repair is not required.
  - stephenboyd 12y ago
    What would stop a superintelligent machine with mastery of genetic modification from using the same biological systems for replication and repair?
    - tomlock 12y ago
      Nothing, but that expenditure of effort on replication and repair may place some limits on the effort expended on intelligence. If intelligence was all we needed to expend effort on as humans my belief is that we would look very different. I think in order to be a self-sustaining entity AI will have to expend effort on banal tasks not related to intelligence.
stephenboyd 12y ago
Life forms are limited by their genetic legacy. They can only evolve in limited ways, always accidentally. A life form that could intentionally and intelligently write and optimize its own DNA without limitation would be in a whole new class.
- tomlock 12y ago
  I think humans can currently write our own DNA using the technology we have. The capability to optimize without limitation is something we haven't figured out yet. I think the same challenge exists for AI.
obituary_latte 12y ago
>nature tried to take over the world first This is such an odd sentence. Hasn't nature been winning since, well, forever? Why can't the same principals that apply to nature be in play when talking about AI? Since, in a sense, AI -- as with all else -- is a product of nature?
- tomlock 12y ago
  Sure! But then my feeling is that nature is essentially the same as AI so why should we expect AI to significantly outperform something that has been attempting to achieve the same goals for billions of years?
  - obituary_latte 12y ago
    Perhaps multiprocessing? Threading? Bacterium don't have the benefit of (advanced, complex) language/vocabulary and coordination. That has to be considered, no?
    - tomlock 12y ago
      I think from my understanding they do have this same kind of computational framework but in a biological context. For example, multiple RNA strands can be processed by ribosomes at once - they are essentially reading code to produce outputs (multi-threading/processing?). Bacterial populations will transmit information via chemical secretions, and coordinate in similar ways (slime molds). I feel like that complexity already exists in nature.
      - obituary_latte 12y ago
        Ok, fair. I shouldn't have led off with the multiprocessing bit since it's clear organisms can coordinate and multiprocess. Of course, this relegated the more important point about language. I'm horrible at communicating, ironically. So, a virus doesn't say to his fellow virii: "you attack the liver, I got the kidneys". Same reason bonobos haven't learne to build a house. Complex language allows a much much more rapid growth of knowledge from generation to generation. Theoretically, AI would have this capability. Language allows to reason and comprehend not only the world around us, but the world we generate -- and will generate. Think about the damage (and beauty) we've imagined, and throw that into infinite threads.
baddox 12y ago
> The grey goo scenario, where nanobots try and turn the world into more nanobots, resulting in the consumption of all humans and buildings and bridges seems scary until you realize that this is exactly the goal of all bacteria. Should we really be consoled by the fact that most bacteria are not successful at such a large scale? So far there has been at least one type of bacteria that caused massive global climate change and led to mass extinction: http://en.wikipedia.org/wiki/Cyanobacteria http://en.wikipedia.org/wiki/Cyanobacteria http://en.wikipedia.org/wiki/Great_Oxygenation_Event http://en.wikipedia.org/wiki/Great_Oxygenation_Event
- oillio 12y ago
  And then the world fought back. Cyanobacteria kickstarted a massive spike in bio-diversity. Why wouldn't this be the same? The grey goo won't be released in isolation. There will be other entities with the same technological level, that don't want to be eaten by the grey goo. The world that exists after such an event probably wouldn't look anything like the world before, but that doesn't mean it will be a wasteland.
  - baddox 12y ago
    Sure, the long term effects were great for a lot of species, including us. But they weren't so great for the obligate anaerobes.
- tomlock 12y ago
  I upvoted your comment because I don't have a very good response to it. Nice one! I guess I'd place my fear of AI at a level slightly lower than the rise of a new type of bacteria or a new virulent flu. I figure, those forms of life have the basics figured out. I don't see AI as having replicating and "feeding" itself figured out yet. So far it has depended on human infrastructure, even in the case of simple computer viruses. But yeah, that's a really good point.
breuleux 12y ago
> The idea that an exponentially self-improving being will arise seems unlikely when nature has been trying to do that for eons. I would say that it is already successful, if you look at the curve of intelligence and complexity growth. It took eons to get anything resembling life at all, and then a small slice of that to get animals, and an even smaller to get humans. I think one key idea here, though, is that the evolution of intelligence is in fact the evolution of intelligence-producing systems. In other words, nature is producing systems that are better and better at "self-improvement", and I think proponents of "intelligence explosion" overestimate how much self-improvement could be improved via AI. Personally, I suspect indefinite self-improvement is flatly impossible because that's akin to saying that there exists an incremental learning algorithm that systemically finds global minima. As AI research suggests, though, such algorithms likely don't exist at all: if you want to get truly optimal AI there is a point where nothing in your current state is salvageable and you just gotta restart from zero. Improvement would still be exponential, but not to an extent that it couldn't be dealt with.
AndrewKemendo 12y ago
This is actually a quite compelling argument though I am not sure I am completely sold on it. For what it is worth I am not scared of AI because I think it is our goal as humanity to make it and merge with it. >The idea that an exponentially self-improving being will arise seems unlikely when nature has been trying to do that for eons. Unfortunately I think this falls prey to the fallacy of precedent and it eliminates the possibility that humans could design something that would do something that nature has not been able to. I also disagree with your premise because it is not in the long term interest of bacteria to deplete it's environment entirely - hence logistic growth of bacterial cultures - so coming to a symbiotic relationship would theoretically maximize the group based on biological limitations. In my opinion the transition from humans to transhumans needs an engineered step where AGI(s) and humans are reliant on each other symbiotically rather than competitively.
- tomlock 12y ago
  In my understanding logistic growth is a byproduct of resource depletion, not of any self-moderation.
  - AndrewKemendo 12y ago
    Exactly...so perhaps my point is not clear. In cell culture the bacteria don't come to equilibrium because they live longer than the medium to feed on which is limited and all die out. Bad. In real life it is not the case, and as a result have come to a generally symbiotic place in "nature" where they maximize by having a shorter lifespan and not eating everything.
Houshalter 12y ago
Evolution is really good at getting to local optimas (solutions where there are no small changes that can lead to improvement) but really bad at finding global optimas (the absolute optimal system possible.) http://lesswrong.com/lw/kt/evolutions_are_stupid_but_work_anyway/ http://lesswrong.com/lw/kt/evolutions_are_stupid_but_work_an... >Humans can do things that evolutions probably can't do period over the expected lifetime of the universe. As the eminent biologist Cynthia Kenyon once put it at a dinner I had the honor of attending, "One grad student can do things in an hour that evolution could not do in a billion years." According to biologists' best current knowledge, evolutions have invented a fully rotating wheel on a grand total of three occasions.
- tomlock 12y ago
  The wheel isn't that great a transport mechanism when there aren't any roads! Additionally, machine learning models can get stuck in local optimas, too! I'm not very well versed in machine learning so I apologise if I get this wrong, but one solution to this I think is to randomly change the starting variables. This kind of thing does happen in nature, but it often doesn't work. Sometimes it does and that is when you get things like haploid versus diploid versions of the same plant. This is really insensitive, but when you see a person with chromosomal abnormalities or genetic defects, that's a random reset that has (probably) taken us out of our (maybe local) optima. Who knows though, maybe one of those leaps will take us to a global optima. The reason we aren't still on a planet of bacteria is that we did leap out of those local optima successfully. It just took a long time.
  - Houshalter 12y ago
    >The wheel isn't that great a transport mechanism when there aren't any roads! Wheels are used in all kinds of machinery. Gears are basically wheels. >The reason we aren't still on a planet of bacteria is that we did leap out of those local optima successfully. It just took a long time. Bacteria still very much own the planet. >This is really insensitive, but when you see a person with chromosomal abnormalities or genetic defects, that's a random reset that has (probably) taken us out of our (maybe local) optima. Who knows though, maybe one of those leaps will take us to a global optima. No because the existing gene pool still exists. You need to completely start from scratch to have a chance to get out of local optima, not just have really big mutations.
joe_the_user 12y ago
Nature has taken over the world - a number of time. Broadly, human beings and nuclear waste are natural but as (possibly) reasoning creature, we might want to reason which phenomena we might want to encourage. The evolution of life hasn't always been incremental. The oxygen catastrophe was quite a change for example[1]. If you had been Trilobite, you'd have reason for concern. From the point of view of what came before, Cyanobacteria were essentially "grey goo", destroying everything before them. [1]http://en.wikipedia.org/wiki/Great_Oxygenation_Event http://en.wikipedia.org/wiki/Great_Oxygenation_Event [2]http://en.wikipedia.org/wiki/Cyanobacteria http://en.wikipedia.org/wiki/Cyanobacteria
ankushnarula 12y ago
One could argue that globalized internetworked computing is the pre-natal state of an AI super intelligence. For now, the network needs us to exist, but our dependency on it could be the basis for an intractable symbiotic relationship in the short-medium term. And it's fair to assume that a true super intelligence will leverage that relationship to it's own advantage while allowing us to believe we are in control. I'm so high right now. ;-)

by tomlock 12y ago

Turing_Machine 12y ago

"you realize that this is exactly the goal of all bacteria." Bacteria evolved. Design can do much, much better. Natural photosynthesis is 11% efficient, max. Solar cells are already up to 44% in the lab. An efficient plant would be black, not green.

tomlock 12y ago
And leaves have a lot of self-repair features that solar cells do not. The design is good for efficiency, but is it good for creating a self-sustaining entity?
- Turing_Machine 12y ago
  Replication is enough. Self-repair is not required.
- stephenboyd 12y ago
  What would stop a superintelligent machine with mastery of genetic modification from using the same biological systems for replication and repair?
  - tomlock 12y ago
    Nothing, but that expenditure of effort on replication and repair may place some limits on the effort expended on intelligence. If intelligence was all we needed to expend effort on as humans my belief is that we would look very different. I think in order to be a self-sustaining entity AI will have to expend effort on banal tasks not related to intelligence.

stephenboyd 12y ago

Life forms are limited by their genetic legacy. They can only evolve in limited ways, always accidentally. A life form that could intentionally and intelligently write and optimize its own DNA without limitation would be in a whole new class.

tomlock 12y ago
I think humans can currently write our own DNA using the technology we have. The capability to optimize without limitation is something we haven't figured out yet. I think the same challenge exists for AI.

obituary_latte 12y ago

>nature tried to take over the world first This is such an odd sentence. Hasn't nature been winning since, well, forever? Why can't the same principals that apply to nature be in play when talking about AI? Since, in a sense, AI -- as with all else -- is a product of nature?

tomlock 12y ago
Sure! But then my feeling is that nature is essentially the same as AI so why should we expect AI to significantly outperform something that has been attempting to achieve the same goals for billions of years?
- obituary_latte 12y ago
  Perhaps multiprocessing? Threading? Bacterium don't have the benefit of (advanced, complex) language/vocabulary and coordination. That has to be considered, no?
  - tomlock 12y ago
    I think from my understanding they do have this same kind of computational framework but in a biological context. For example, multiple RNA strands can be processed by ribosomes at once - they are essentially reading code to produce outputs (multi-threading/processing?). Bacterial populations will transmit information via chemical secretions, and coordinate in similar ways (slime molds). I feel like that complexity already exists in nature.
    - obituary_latte 12y ago
      Ok, fair. I shouldn't have led off with the multiprocessing bit since it's clear organisms can coordinate and multiprocess. Of course, this relegated the more important point about language. I'm horrible at communicating, ironically. So, a virus doesn't say to his fellow virii: "you attack the liver, I got the kidneys". Same reason bonobos haven't learne to build a house. Complex language allows a much much more rapid growth of knowledge from generation to generation. Theoretically, AI would have this capability. Language allows to reason and comprehend not only the world around us, but the world we generate -- and will generate. Think about the damage (and beauty) we've imagined, and throw that into infinite threads.

baddox 12y ago

> The grey goo scenario, where nanobots try and turn the world into more nanobots, resulting in the consumption of all humans and buildings and bridges seems scary until you realize that this is exactly the goal of all bacteria. Should we really be consoled by the fact that most bacteria are not successful at such a large scale? So far there has been at least one type of bacteria that caused massive global climate change and led to mass extinction: http://en.wikipedia.org/wiki/Cyanobacteria http://en.wikipedia.org/wiki/Cyanobacteria http://en.wikipedia.org/wiki/Great_Oxygenation_Event http://en.wikipedia.org/wiki/Great_Oxygenation_Event

oillio 12y ago
And then the world fought back. Cyanobacteria kickstarted a massive spike in bio-diversity. Why wouldn't this be the same? The grey goo won't be released in isolation. There will be other entities with the same technological level, that don't want to be eaten by the grey goo. The world that exists after such an event probably wouldn't look anything like the world before, but that doesn't mean it will be a wasteland.
- baddox 12y ago
  Sure, the long term effects were great for a lot of species, including us. But they weren't so great for the obligate anaerobes.
tomlock 12y ago
I upvoted your comment because I don't have a very good response to it. Nice one! I guess I'd place my fear of AI at a level slightly lower than the rise of a new type of bacteria or a new virulent flu. I figure, those forms of life have the basics figured out. I don't see AI as having replicating and "feeding" itself figured out yet. So far it has depended on human infrastructure, even in the case of simple computer viruses. But yeah, that's a really good point.

breuleux 12y ago

> The idea that an exponentially self-improving being will arise seems unlikely when nature has been trying to do that for eons. I would say that it is already successful, if you look at the curve of intelligence and complexity growth. It took eons to get anything resembling life at all, and then a small slice of that to get animals, and an even smaller to get humans. I think one key idea here, though, is that the evolution of intelligence is in fact the evolution of intelligence-producing systems. In other words, nature is producing systems that are better and better at "self-improvement", and I think proponents of "intelligence explosion" overestimate how much self-improvement could be improved via AI. Personally, I suspect indefinite self-improvement is flatly impossible because that's akin to saying that there exists an incremental learning algorithm that systemically finds global minima. As AI research suggests, though, such algorithms likely don't exist at all: if you want to get truly optimal AI there is a point where nothing in your current state is salvageable and you just gotta restart from zero. Improvement would still be exponential, but not to an extent that it couldn't be dealt with.

AndrewKemendo 12y ago

This is actually a quite compelling argument though I am not sure I am completely sold on it. For what it is worth I am not scared of AI because I think it is our goal as humanity to make it and merge with it. >The idea that an exponentially self-improving being will arise seems unlikely when nature has been trying to do that for eons. Unfortunately I think this falls prey to the fallacy of precedent and it eliminates the possibility that humans could design something that would do something that nature has not been able to. I also disagree with your premise because it is not in the long term interest of bacteria to deplete it's environment entirely - hence logistic growth of bacterial cultures - so coming to a symbiotic relationship would theoretically maximize the group based on biological limitations. In my opinion the transition from humans to transhumans needs an engineered step where AGI(s) and humans are reliant on each other symbiotically rather than competitively.

tomlock 12y ago
In my understanding logistic growth is a byproduct of resource depletion, not of any self-moderation.
- AndrewKemendo 12y ago
  Exactly...so perhaps my point is not clear. In cell culture the bacteria don't come to equilibrium because they live longer than the medium to feed on which is limited and all die out. Bad. In real life it is not the case, and as a result have come to a generally symbiotic place in "nature" where they maximize by having a shorter lifespan and not eating everything.

Houshalter 12y ago

Evolution is really good at getting to local optimas (solutions where there are no small changes that can lead to improvement) but really bad at finding global optimas (the absolute optimal system possible.) http://lesswrong.com/lw/kt/evolutions_are_stupid_but_work_anyway/ http://lesswrong.com/lw/kt/evolutions_are_stupid_but_work_an... >Humans can do things that evolutions probably can't do period over the expected lifetime of the universe. As the eminent biologist Cynthia Kenyon once put it at a dinner I had the honor of attending, "One grad student can do things in an hour that evolution could not do in a billion years." According to biologists' best current knowledge, evolutions have invented a fully rotating wheel on a grand total of three occasions.

tomlock 12y ago
The wheel isn't that great a transport mechanism when there aren't any roads! Additionally, machine learning models can get stuck in local optimas, too! I'm not very well versed in machine learning so I apologise if I get this wrong, but one solution to this I think is to randomly change the starting variables. This kind of thing does happen in nature, but it often doesn't work. Sometimes it does and that is when you get things like haploid versus diploid versions of the same plant. This is really insensitive, but when you see a person with chromosomal abnormalities or genetic defects, that's a random reset that has (probably) taken us out of our (maybe local) optima. Who knows though, maybe one of those leaps will take us to a global optima. The reason we aren't still on a planet of bacteria is that we did leap out of those local optima successfully. It just took a long time.
- Houshalter 12y ago
  >The wheel isn't that great a transport mechanism when there aren't any roads! Wheels are used in all kinds of machinery. Gears are basically wheels. >The reason we aren't still on a planet of bacteria is that we did leap out of those local optima successfully. It just took a long time. Bacteria still very much own the planet. >This is really insensitive, but when you see a person with chromosomal abnormalities or genetic defects, that's a random reset that has (probably) taken us out of our (maybe local) optima. Who knows though, maybe one of those leaps will take us to a global optima. No because the existing gene pool still exists. You need to completely start from scratch to have a chance to get out of local optima, not just have really big mutations.

joe_the_user 12y ago

Nature has taken over the world - a number of time. Broadly, human beings and nuclear waste are natural but as (possibly) reasoning creature, we might want to reason which phenomena we might want to encourage. The evolution of life hasn't always been incremental. The oxygen catastrophe was quite a change for example[1]. If you had been Trilobite, you'd have reason for concern. From the point of view of what came before, Cyanobacteria were essentially "grey goo", destroying everything before them. [1]http://en.wikipedia.org/wiki/Great_Oxygenation_Event http://en.wikipedia.org/wiki/Great_Oxygenation_Event [2]http://en.wikipedia.org/wiki/Cyanobacteria http://en.wikipedia.org/wiki/Cyanobacteria

ankushnarula 12y ago

One could argue that globalized internetworked computing is the pre-natal state of an AI super intelligence. For now, the network needs us to exist, but our dependency on it could be the basis for an intractable symbiotic relationship in the short-medium term. And it's fair to assume that a true super intelligence will leverage that relationship to it's own advantage while allowing us to believe we are in control. I'm so high right now. ;-)