3 ms·

I'm not sure I understand how exactly a human-equivalent AGI would be more dangerous than an actual human.

by atroyn 11y ago

lossolo 11y ago
We are constrained by organic capabilities of our brains. AGI will not be. It will see things we can't, it will discover things which would take us centuries on our own. It's analytical capabilities will be unmatched for any group of humans. If hitler would have real AGI, there would be only one country in the world now.
- atroyn 11y ago
  Consider: by definition, the human brain has the equivalent 'intelligence' of a human-equivalent A.I. That means we are able to understand the A.I precisely as well as it can understand itself. If the A.I can self-improve, then we can also improve it. We're still equivalent. Hitler had literally millions of human-equivalent-intelligences - they even had excellent sensor and actuator suites - and it didn't work out so well for him in any case.
  - mediumdeviation 11y ago
    Because a human brain is not field programmable, and improving cognitive ability isn't as simple as plugging in another stick of RAM. The worry about AI is the bootstrap scenario, where an AI gains sufficient intelligence to modify and upgrade itself. Sure, you're right in that humans don't understand enough about our own brains to perform such modifications, but that may simply be down to a lack of knowledge, rather than not having sufficient cognitive ability to comprehend the brain. Additionally the hardware which an AI can run on is generally assumed to be significantly more malleable than our limited gray matter confined within our cranium
    - atroyn 11y ago
      I think the underlying assumption you're making is that a sufficiently powerful computer could emulate the chemical processes in the brain, and that this is therefore equivalent to running a brain on a computer, decoupling cognition from brain meat. The problem is there are hard physical limits you run into in computation - it's more efficient to just do the chemistry than it is to simulate it. Our meat brains might be more efficient in terms of density and power efficiency than any physically feasible digital computer. Further, computing power can't be increased arbitrarily, especially when networks and switching come into play. Even if your simulated brain worked out how to improve itself, the resource cost to actually do so might be prohibitively high.
  - deleted 11y ago
    [deleted]
mattmanser 11y ago
Because it can think so much quicker, so a few seconds will be generations in their time. And on top of that, if they have human intelligence, they'd be able to improve it in those few seconds (because we had and they'd have access to our research), make smarter AI, and on and on exponentially fast.
- atroyn 11y ago
  I am not sure I agree that it would necessarily be quicker. Evolutionarily speaking, a rat's brain's neurons switch just as fast as a humans. The number and density of neurons seem to define the level of cognition, not the switching speed. I have a hunch that the constraint on switching speed actually results in intelligence rather than constrains it. Consider seizures in human brains for example. Secondly, consider that humans have had the same brains for millennia and yet we still cannot improve them. We barely understand the first thing about them. I don't see how making the process faster, at the same level of cognition, would help.
jamii 11y ago
There are a lot of assumptions you have to make to get to the point where it doesn't have the capability to be dangerous: * No improvement beyond human level (correlation between IQ and job-performance is variously estimated at around 0.5 and IQ largely comes down to executive function and working memory - so we have to assume an AI design that doesn't allow adding more working memory for some reason) * No extra individual capabilities (eg human-level but with mental access to Google Scholar) * No extra group capabilities (eg human-level but with telepathy between AIs - look at the difference the internet is making to human productivity eg Linux) * No horizontal scaling over time (eg one human-level AI now, one million once Moores law kicks in - that's a lot of scientists/strategists/politicians/advertisers) So in the situation where we make an AI that is somehow structurally limited to human-level intelligence and doesn't benefit from extra interfaces and we somehow can't afford to ever make many of them, then, sure, I can't think of a way that it could be dangerous. In a few minutes of cursory thought I came up with four assumptions that were necessary for assuming that they won't be more capable than humans. What are the odds that all of those hold? [EDIT isn't dangerous -> doesn't have the capability to be dangerous]
- cstross 11y ago
  Those four assumptions have nothing to do with danger and everything to do with capability. I think a more interesting question is "how do we make an intelligence, artificial or human, that isn't potentially dangerous?" Humans have a bunch of low-level primate aggression that seems to be wired in; we can modulate it (we're largely self-domesticating) but we can't totally get rid of it and more importantly, we can't look at a baby's genome (and nurturing environment) and say "yup, this one's going to grow up to be Hitler", or "this 'un's going to be a pacifist, altruist, and general benefactor". On the other hand we can probably add a bunch of monitoring code to any AI we develop rather than grow: if nothing else, consider the possibility of simulation runs looking for an empathy deficiency.
  - jamii 11y ago
    You're right that the interesting question is how to make a friendly intelligence, but all the groups who are trying to draw attention to that problem are running into the wall of doubt about capability. The OP said: > I'm not sure I understand how exactly a human-equivalent AGI would be more dangerous than an actual human. Since an actual human might be out to get me already the question is not motives but capability - should I fear evil AIs more than I fear evil humans? Or an evil human with a friendly leashed AI? The argument typically revolves around whether or not AI would self-improve and lead to runaway intelligence, but I think that that's a distraction - there are plenty of other ways AI could be dangerous that are easier to persuade people with without running into the that-sounds-like-scifi filter.

I'm not sure I understand how exactly a human-equivalent AGI would be more dangerous than an actual human.

by atroyn 11y ago

I'm not sure I understand how exactly a human-equivalent AGI would be more dangerous than an actual human.

We are constrained by organic capabilities of our brains. AGI will not be. It will see things we can't, it will discover things which would take us centuries on our own. It's analytical capabilities will be unmatched for any group of humans. If hitler would have real AGI, there would be only one country in the world now.

atroyn 11y ago
Consider: by definition, the human brain has the equivalent 'intelligence' of a human-equivalent A.I. That means we are able to understand the A.I precisely as well as it can understand itself. If the A.I can self-improve, then we can also improve it. We're still equivalent. Hitler had literally millions of human-equivalent-intelligences - they even had excellent sensor and actuator suites - and it didn't work out so well for him in any case.
- mediumdeviation 11y ago
  Because a human brain is not field programmable, and improving cognitive ability isn't as simple as plugging in another stick of RAM. The worry about AI is the bootstrap scenario, where an AI gains sufficient intelligence to modify and upgrade itself. Sure, you're right in that humans don't understand enough about our own brains to perform such modifications, but that may simply be down to a lack of knowledge, rather than not having sufficient cognitive ability to comprehend the brain. Additionally the hardware which an AI can run on is generally assumed to be significantly more malleable than our limited gray matter confined within our cranium
  - atroyn 11y ago
    I think the underlying assumption you're making is that a sufficiently powerful computer could emulate the chemical processes in the brain, and that this is therefore equivalent to running a brain on a computer, decoupling cognition from brain meat. The problem is there are hard physical limits you run into in computation - it's more efficient to just do the chemistry than it is to simulate it. Our meat brains might be more efficient in terms of density and power efficiency than any physically feasible digital computer. Further, computing power can't be increased arbitrarily, especially when networks and switching come into play. Even if your simulated brain worked out how to improve itself, the resource cost to actually do so might be prohibitively high.
- deleted 11y ago
  [deleted]

mattmanser 11y ago

Because it can think so much quicker, so a few seconds will be generations in their time. And on top of that, if they have human intelligence, they'd be able to improve it in those few seconds (because we had and they'd have access to our research), make smarter AI, and on and on exponentially fast.

atroyn 11y ago
I am not sure I agree that it would necessarily be quicker. Evolutionarily speaking, a rat's brain's neurons switch just as fast as a humans. The number and density of neurons seem to define the level of cognition, not the switching speed. I have a hunch that the constraint on switching speed actually results in intelligence rather than constrains it. Consider seizures in human brains for example. Secondly, consider that humans have had the same brains for millennia and yet we still cannot improve them. We barely understand the first thing about them. I don't see how making the process faster, at the same level of cognition, would help.

jamii 11y ago

There are a lot of assumptions you have to make to get to the point where it doesn't have the capability to be dangerous: * No improvement beyond human level (correlation between IQ and job-performance is variously estimated at around 0.5 and IQ largely comes down to executive function and working memory - so we have to assume an AI design that doesn't allow adding more working memory for some reason) * No extra individual capabilities (eg human-level but with mental access to Google Scholar) * No extra group capabilities (eg human-level but with telepathy between AIs - look at the difference the internet is making to human productivity eg Linux) * No horizontal scaling over time (eg one human-level AI now, one million once Moores law kicks in - that's a lot of scientists/strategists/politicians/advertisers) So in the situation where we make an AI that is somehow structurally limited to human-level intelligence and doesn't benefit from extra interfaces and we somehow can't afford to ever make many of them, then, sure, I can't think of a way that it could be dangerous. In a few minutes of cursory thought I came up with four assumptions that were necessary for assuming that they won't be more capable than humans. What are the odds that all of those hold? [EDIT isn't dangerous -> doesn't have the capability to be dangerous]

cstross 11y ago
Those four assumptions have nothing to do with danger and everything to do with capability. I think a more interesting question is "how do we make an intelligence, artificial or human, that isn't potentially dangerous?" Humans have a bunch of low-level primate aggression that seems to be wired in; we can modulate it (we're largely self-domesticating) but we can't totally get rid of it and more importantly, we can't look at a baby's genome (and nurturing environment) and say "yup, this one's going to grow up to be Hitler", or "this 'un's going to be a pacifist, altruist, and general benefactor". On the other hand we can probably add a bunch of monitoring code to any AI we develop rather than grow: if nothing else, consider the possibility of simulation runs looking for an empathy deficiency.
- jamii 11y ago
  You're right that the interesting question is how to make a friendly intelligence, but all the groups who are trying to draw attention to that problem are running into the wall of doubt about capability. The OP said: > I'm not sure I understand how exactly a human-equivalent AGI would be more dangerous than an actual human. Since an actual human might be out to get me already the question is not motives but capability - should I fear evil AIs more than I fear evil humans? Or an evil human with a friendly leashed AI? The argument typically revolves around whether or not AI would self-improve and lead to runaway intelligence, but I think that that's a distraction - there are plenty of other ways AI could be dangerous that are easier to persuade people with without running into the that-sounds-like-scifi filter.