3 ms·

We don’t produce intelligent humans nature does. Otherwise I agree. We’re learning what the end result of intelligence looks like (predictions) not the actual

We don’t produce intelligent humans nature does. Otherwise I agree. We’re learning what the end result of intelligence looks like (predictions) not the actual intelligence yet. The intelligence is what determines the shape of these networks so they can even learn a useful pattern. That’s still gonna be humans for the foreseeable future.

naasking 4y ago
Yes, nature produced intelligence while optimizing only for reproductive fitness. In other words, intelligence emerged spontaneously while solving a seemingly completely unrelated problem. That should terrify you. Current AI research largely isn't focused on general intelligence, but the possibility remains that it could still spontaneously emerge from it. We can't even quantify how likely that is because we don't understand intelligence, so whatever intuition you have about this likelihood it's completely meaningless and not based on any meaningful data. We're in uncharted waters.
- dekhn 4y ago
  That intelligence is a emergent property of reproductive fitness isn't terrifying- it's amazing and exciting and suggests a wide range of scientific opportunities. For a while I ran a project that ran a (scientific) binary on every Google machine in production using all the idle cores on the machine to do protein design (it worked!). At the end of the project we seriously considered making the binary instead a learning system that had access to 'sensors' (IE, signals from Google production machines) and 'outputs' (the ability to change production parameters) and just let that run autonomously. I figured Google prod is one of the few computational/networking systems with enough inherent complexity and resources that spontaneously emergent general intelligence was not inconceivable. However, the project was deemed too risky by SRE, rightfully so. Not because of emergent intelligence but emergent disruption to serving ads.
  - naasking 4y ago
    > That intelligence is a emergent property of reproductive fitness isn't terrifying You missed the terrifying aspect. It's not terrifying that general intelligence emerged from optimizing for reproductive fitness specifically, it's terrifying that general intelligence emerged while nature was stumbling around randomly in the dark trying to optimize for a complete unrelated property. Guess what we're doing with machine learning?
    - edgyquant 4y ago
      This is a strange comparison that doesn’t seem logical to me. If we created intelligences and set them in competition with each other we may get something to evolve millions of years later. General intelligence didn’t poof into existence it happened over half a billion years and even then it came from fungi which is itself an intelligence.
      - naasking 4y ago
        > If we created intelligences and set them in competition with each other we may get something to evolve millions of years later. Why millions of years? Do you agree or disagree that humans can develop technology faster than nature could evolve it on its own? It took maybe 70 years for computers to go from not existing, to matching humans on visual recognition tasks. As you just acknowledged, it took nature at least billions of years to evolve humans. Is not focused technological development obviously orders of magnitude faster at evolving intelligence than nature? Does it not then follow that artificial general intelligence is a lot closer than an argument based on natural evolution might imply? What exactly is not logical about this argument?
      - itishappy 4y ago
        It happened over billions of generations, and an AI generation can be millions of times shorter than a biological one. One of those is also shrinking rapidly.
      - dekhn 4y ago
        Minor technical correction- it would be false to say that GI came from fungi as we humans have GI, but don't descend from fungi (we share a common ancestor) and likely haven't received the genes for GI (horizontal gene transfer) from them. Personally I figure that once you have a complex enough system with feedback regulation and self-preservation, something like general intelligence occurs spontaneously through the normal mechanisms of evolution (probably more than once in different kingdoms) because it provides a heritable survival edge.

naasking 4y ago

Yes, nature produced intelligence while optimizing only for reproductive fitness. In other words, intelligence emerged spontaneously while solving a seemingly completely unrelated problem. That should terrify you. Current AI research largely isn't focused on general intelligence, but the possibility remains that it could still spontaneously emerge from it. We can't even quantify how likely that is because we don't understand intelligence, so whatever intuition you have about this likelihood it's completely meaningless and not based on any meaningful data. We're in uncharted waters.

dekhn 4y ago
That intelligence is a emergent property of reproductive fitness isn't terrifying- it's amazing and exciting and suggests a wide range of scientific opportunities. For a while I ran a project that ran a (scientific) binary on every Google machine in production using all the idle cores on the machine to do protein design (it worked!). At the end of the project we seriously considered making the binary instead a learning system that had access to 'sensors' (IE, signals from Google production machines) and 'outputs' (the ability to change production parameters) and just let that run autonomously. I figured Google prod is one of the few computational/networking systems with enough inherent complexity and resources that spontaneously emergent general intelligence was not inconceivable. However, the project was deemed too risky by SRE, rightfully so. Not because of emergent intelligence but emergent disruption to serving ads.
- naasking 4y ago
  > That intelligence is a emergent property of reproductive fitness isn't terrifying You missed the terrifying aspect. It's not terrifying that general intelligence emerged from optimizing for reproductive fitness specifically, it's terrifying that general intelligence emerged while nature was stumbling around randomly in the dark trying to optimize for a complete unrelated property. Guess what we're doing with machine learning?
  - edgyquant 4y ago
    This is a strange comparison that doesn’t seem logical to me. If we created intelligences and set them in competition with each other we may get something to evolve millions of years later. General intelligence didn’t poof into existence it happened over half a billion years and even then it came from fungi which is itself an intelligence.
    - naasking 4y ago
      > If we created intelligences and set them in competition with each other we may get something to evolve millions of years later. Why millions of years? Do you agree or disagree that humans can develop technology faster than nature could evolve it on its own? It took maybe 70 years for computers to go from not existing, to matching humans on visual recognition tasks. As you just acknowledged, it took nature at least billions of years to evolve humans. Is not focused technological development obviously orders of magnitude faster at evolving intelligence than nature? Does it not then follow that artificial general intelligence is a lot closer than an argument based on natural evolution might imply? What exactly is not logical about this argument?
    - itishappy 4y ago
      It happened over billions of generations, and an AI generation can be millions of times shorter than a biological one. One of those is also shrinking rapidly.
    - dekhn 4y ago
      Minor technical correction- it would be false to say that GI came from fungi as we humans have GI, but don't descend from fungi (we share a common ancestor) and likely haven't received the genes for GI (horizontal gene transfer) from them. Personally I figure that once you have a complex enough system with feedback regulation and self-preservation, something like general intelligence occurs spontaneously through the normal mechanisms of evolution (probably more than once in different kingdoms) because it provides a heritable survival edge.