5 ms·

> You can indeed use backprop to train a separate learning rule superior to naïve backprop. I used to dismiss the idea of an impending singularity. Now I'm not

by scribu 6y ago

> You can indeed use backprop to train a separate learning rule superior to naïve backprop. I used to dismiss the idea of an impending singularity. Now I'm not so sure. Hopefully AGIs will reach hard physical limits to self-improvement, before taking over the world.

jjk166 6y ago
Given the obvious benefits of increased intelligence, the fact that hominid brain size (and presumably computational power) plateaued for the past 300,000 or so years, and that no other species has developed superior intelligence (ie there are bigger but not more effective brains out there), does seem to indicate that we are at or close to a local maxima for biological intelligence. Presumably that's some threshold beyond which the gains from increased computing power lead to limited improvements in intelligence. Of course that's not to say it's a global maximum.
- semi-extrinsic 6y ago
  That, or our brains have hit some biological equivalent of Moore's law ending for silicon computing. Maybe getting a doubling in brain size would require a quadrupling in brain energy consumption (and power dissipation) at our level.
  - jjk166 6y ago
    Yeah that's what I mean by not scaling above a threshold. Our brains could be bigger (neanderthals had larger ones than us), but those bigger brains for whatever reason weren't "worth it."
- mrmonkeyman 6y ago
  For billions of years we did not exist. The organisms then could have said the same, about being a local maxima.
- danielheath 6y ago
  In order to get brains as big as we already do, biology has us: * Birthing prematurely compared to other species (most species live young are functional at birth) in order to fit the brain through the hips. * Permanently reducing the mothers top speed (wider hips means the joints are exposed to higher forces while walking). * Requiring 5% or so more food. All three of those have been serious impediments to human survival for most of our existence, and provided a counter-pressure against improvements in biological intelligence. In the past century, the Haber-Bosch process has meant we can reliably grow enough food without pursuit hunting. Without that counterpressure, evolutionary improvement seems very likely, but will take hundreds of thousands of years without genetic engineering or eugenics (neither of which seems like a good idea to me).
  - mantap 6y ago
    Intelligence has a strong nurture component that we call education. It's unclear what good, if any, better genetics would do. We are already capable of reproducing and outsourcing many of the brain's functions to computers, and it seems likely we will reach the AGI endgame within a century or two conservatively. Arguably we have far too much intelligence and will end up causing our own destruction.
  - Mumps 6y ago
    FYI: the obstetrical dilemma isn't so strongly supported nowadays [0]. I can't find the study i'm thinking of at the moment, but at least one study proposes the early birthing in humans allows the brain extra social development. [0] https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0118903 https://journals.plos.org/plosone/article?id=10.1371/journal...
    - danielheath 6y ago
      That link says wider hips don't slow down walking. However, women need hip replacements at a far higher rate than men. Given men are typically heavier one would expect their joints to wear out faster, but as alluded in my comment upthread, widening the pelvis means walking exerts a higher peak force on the hip joint due to increased torque, which is a plausible mechanism for it wearing out faster.
- zepto 6y ago
  > and that no other species has developed superior intelligence Another possibility is that other species are constrained by us, and we have manipulated our environment to reduce the survival advantage of increased intelligence for humans as a whole.
  - jjk166 6y ago
    We've only been able to do so for a brief sliver of geological time. That earth would go billions of years without advanced intelligence only to have two such intelligences emerge independently within a few thousand years of eachother seems improbable.
ilaksh 6y ago
I've been interested in AGI for a long time. But I've never been on board with the speculation about AGIs making multiple orders of magnitude improvement in intelligence. That is total speculation, and also it's not necessary to assume so much to have a similar outcome. It seems much easier to imagine an AI that is 2 or 3 times smarter than a human, and has very fast transfer of knowledge with other compatible beings. I think that's enough for them to take over, if there are enough of them. But anyway it seems obvious to me that we absolutely should avoid trying to build fully autonomous digital creatures that compete with us. We should rather aim carefully for something more like a Star Trek computer, without any real autonomy or necessarily fully generalized skills or cognition or animal-like characteristics/drives.

by scribu 6y ago

jjk166 6y ago

Given the obvious benefits of increased intelligence, the fact that hominid brain size (and presumably computational power) plateaued for the past 300,000 or so years, and that no other species has developed superior intelligence (ie there are bigger but not more effective brains out there), does seem to indicate that we are at or close to a local maxima for biological intelligence. Presumably that's some threshold beyond which the gains from increased computing power lead to limited improvements in intelligence. Of course that's not to say it's a global maximum.

semi-extrinsic 6y ago
That, or our brains have hit some biological equivalent of Moore's law ending for silicon computing. Maybe getting a doubling in brain size would require a quadrupling in brain energy consumption (and power dissipation) at our level.
- jjk166 6y ago
  Yeah that's what I mean by not scaling above a threshold. Our brains could be bigger (neanderthals had larger ones than us), but those bigger brains for whatever reason weren't "worth it."
mrmonkeyman 6y ago
For billions of years we did not exist. The organisms then could have said the same, about being a local maxima.
danielheath 6y ago
In order to get brains as big as we already do, biology has us: * Birthing prematurely compared to other species (most species live young are functional at birth) in order to fit the brain through the hips. * Permanently reducing the mothers top speed (wider hips means the joints are exposed to higher forces while walking). * Requiring 5% or so more food. All three of those have been serious impediments to human survival for most of our existence, and provided a counter-pressure against improvements in biological intelligence. In the past century, the Haber-Bosch process has meant we can reliably grow enough food without pursuit hunting. Without that counterpressure, evolutionary improvement seems very likely, but will take hundreds of thousands of years without genetic engineering or eugenics (neither of which seems like a good idea to me).
- mantap 6y ago
  Intelligence has a strong nurture component that we call education. It's unclear what good, if any, better genetics would do. We are already capable of reproducing and outsourcing many of the brain's functions to computers, and it seems likely we will reach the AGI endgame within a century or two conservatively. Arguably we have far too much intelligence and will end up causing our own destruction.
- Mumps 6y ago
  FYI: the obstetrical dilemma isn't so strongly supported nowadays [0]. I can't find the study i'm thinking of at the moment, but at least one study proposes the early birthing in humans allows the brain extra social development. [0] https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0118903 https://journals.plos.org/plosone/article?id=10.1371/journal...
  - danielheath 6y ago
    That link says wider hips don't slow down walking. However, women need hip replacements at a far higher rate than men. Given men are typically heavier one would expect their joints to wear out faster, but as alluded in my comment upthread, widening the pelvis means walking exerts a higher peak force on the hip joint due to increased torque, which is a plausible mechanism for it wearing out faster.
zepto 6y ago
> and that no other species has developed superior intelligence Another possibility is that other species are constrained by us, and we have manipulated our environment to reduce the survival advantage of increased intelligence for humans as a whole.
- jjk166 6y ago
  We've only been able to do so for a brief sliver of geological time. That earth would go billions of years without advanced intelligence only to have two such intelligences emerge independently within a few thousand years of eachother seems improbable.

ilaksh 6y ago

I've been interested in AGI for a long time. But I've never been on board with the speculation about AGIs making multiple orders of magnitude improvement in intelligence. That is total speculation, and also it's not necessary to assume so much to have a similar outcome. It seems much easier to imagine an AI that is 2 or 3 times smarter than a human, and has very fast transfer of knowledge with other compatible beings. I think that's enough for them to take over, if there are enough of them. But anyway it seems obvious to me that we absolutely should avoid trying to build fully autonomous digital creatures that compete with us. We should rather aim carefully for something more like a Star Trek computer, without any real autonomy or necessarily fully generalized skills or cognition or animal-like characteristics/drives.