5 ms·

I'm very interested why you think an advanced AI wouldn't be dangerous? Assume: - AGI wants to stay alive - Humans can create more AIs - Other AIs would compet

by oefnak 4y ago

I'm very interested why you think an advanced AI wouldn't be dangerous? Assume: - AGI wants to stay alive - Humans can create more AIs - Other AIs would compete for the same resources Then: Easiest way to make sure that they would get no more competitors would be...

ryanmcbride 4y ago
AGI and what we're talking about here are completely different.
astrange 4y ago
That’d be sad if we invented a superintelligent AI but still taught it the lump of labor fallacy. Anytime you see “creating an AI will obviously kill you” try reading it as “having children will obviously kill you” and see if it still makes sense.
- alasdair_ 4y ago
  Most children aren't able to recursively improve their own hardware and software in short time spans, and generally most children are unlikely to be many orders of magnitude more intelligent than their parents.
  - astrange 4y ago
    Even in a world where an AI exists, why am I supposed to believe it's going to be able to do any of those things? I do believe it's possible to create one with the same attributes as a human person, it's just anything beyond that is unproven. Rather, it seems like evidence that singularitarianism is actually a religion (https://en.wikipedia.org/wiki/Millenarianism https://en.wikipedia.org/wiki/Millenarianism) which is why it believes things with magic powers will suddenly appear. In particular, exponential growth doesn't exist in nature and always turns into an S-curve… of course it's a problem if it doesn't level out until it's too late.
    - cornel_io 4y ago
      What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? I'd bet a ton that we're nowhere near the top: evolution almost never comes up with the optimal solution for any problem, almost by definition it stops at "meh, good enough to reproduce". And you don't need a ton of intelligence to reproduce. Evolution's sub-optimality is actually one of the strongest arguments against intelligent design, so I'm really hesitant to agree that it requires any sort of leap to estimate that with some actual design it won't be very difficult to blow way past human intelligence once we can get there.
      - astrange 4y ago
        > What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? Well, define "intelligence". People seem to use it in a vague way here - it might be what you call a motte and bailey. The motte (specific definition) is something like "can do math problems really fast" and the bailey is like "high executive function, is always right about everything, can predict the future". For the first one I don't think humans are near a limit, mostly because of the bottleneck in how we get born limiting our head sizes. But it is pretty good if you consider the costs of being alive - food requirements, heat dissipation, being bipedal, surviving being hit on the head, risk of brain cancer, etc, it's done well so far. Similarly an AI is going to have maintenance costs - the more RTX 3090s it runs on, the more calculations it might be able to do, but it's going to have to pay for them and their power bill, and they'll fail or give wrong answers eventually. And where's it getting the money anyway? As for the second kind I don't think you can be exponentially better at it than a human. At least if you are, it's not through intelligence, but it might be through access to more private information, or being rich enough to survive mistakes. As an example, you can't beat the stock market reliably with smarts, but you can by never being forced to sell. The real mystery to me is why people say "AI could recursively improve their own hardware and software in short time spans". I mean, that's clearly a made up concept since none of humans, computers or existing AI do it. But the closest thing I can think of is collective intelligence - humans individually haven't improved in the last 10k years, but we got a lot more humans and conquered everyone else that way. But we're also all individuals competing with each other and paying for our own individual food/maintenance/etc, which makes it different from nodes in an ever-growing AI.
- hackerlight 4y ago
  That's disanalogous, because: - children won't be smarter than any living human. - children have a human brain which makes them predictable (constrained in behavior by current laws, institutions, and most likely a conscience). A better analogy is to ask what happens when a species branches off and evolves into a smarter species, but the dumber ancestor species still exists. I really don't understand the perspective that people in your position take. Is it that you don't think we'll arrive at super intelligent AI, and therefore there's little risk? Or that we will be able to control it? If you think we can control it, why? Like we're not that much smarter than our monkey ancestors and what hope did they ever have of stopping our absolute domination over them? And then all of you call this opinion a "religion" without even explaining why it's wrong?
  - mach1ne 4y ago
    Restricting AI to the lower end of human intelligence (e.g. around IQ of 70) makes it a useful resource which is guaranteed to be safe. A 70 IQ human couldn't take over the world nor disarm any safety features built into their bodies. You're right on the spot regarding the problem of having two different smart species on the same planet. We killed everything between us and chimps. Given enough time, the smarter species can be assumed to always take over.
    - deleted 4y ago
      [deleted]
    - astrange 4y ago
      That’s actually reverse causation - it’s not that we killed “everyone else”, it’s that everyone still alive became “us” after we gave up killing them and interbred with them instead. The British were going around genociding all over the place until pretty recently but now they’ve decided the Irish are human too. Homo sapiens is still expanding even: https://dna-explained.com/2012/11/16/the-new-root-haplogroup-a00/ https://dna-explained.com/2012/11/16/the-new-root-haplogroup...
      - hackerlight 4y ago
namlem 4y ago
AGI could be dangerous, but most people in AI safety are doing nothing useful to mitigate that risk.

by oefnak 4y ago

ryanmcbride 4y ago

AGI and what we're talking about here are completely different.

astrange 4y ago

That’d be sad if we invented a superintelligent AI but still taught it the lump of labor fallacy. Anytime you see “creating an AI will obviously kill you” try reading it as “having children will obviously kill you” and see if it still makes sense.

alasdair_ 4y ago
Most children aren't able to recursively improve their own hardware and software in short time spans, and generally most children are unlikely to be many orders of magnitude more intelligent than their parents.
- astrange 4y ago
  Even in a world where an AI exists, why am I supposed to believe it's going to be able to do any of those things? I do believe it's possible to create one with the same attributes as a human person, it's just anything beyond that is unproven. Rather, it seems like evidence that singularitarianism is actually a religion (https://en.wikipedia.org/wiki/Millenarianism https://en.wikipedia.org/wiki/Millenarianism) which is why it believes things with magic powers will suddenly appear. In particular, exponential growth doesn't exist in nature and always turns into an S-curve… of course it's a problem if it doesn't level out until it's too late.
  - cornel_io 4y ago
    What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? I'd bet a ton that we're nowhere near the top: evolution almost never comes up with the optimal solution for any problem, almost by definition it stops at "meh, good enough to reproduce". And you don't need a ton of intelligence to reproduce. Evolution's sub-optimality is actually one of the strongest arguments against intelligent design, so I'm really hesitant to agree that it requires any sort of leap to estimate that with some actual design it won't be very difficult to blow way past human intelligence once we can get there.
    - astrange 4y ago
      > What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? Well, define "intelligence". People seem to use it in a vague way here - it might be what you call a motte and bailey. The motte (specific definition) is something like "can do math problems really fast" and the bailey is like "high executive function, is always right about everything, can predict the future". For the first one I don't think humans are near a limit, mostly because of the bottleneck in how we get born limiting our head sizes. But it is pretty good if you consider the costs of being alive - food requirements, heat dissipation, being bipedal, surviving being hit on the head, risk of brain cancer, etc, it's done well so far. Similarly an AI is going to have maintenance costs - the more RTX 3090s it runs on, the more calculations it might be able to do, but it's going to have to pay for them and their power bill, and they'll fail or give wrong answers eventually. And where's it getting the money anyway? As for the second kind I don't think you can be exponentially better at it than a human. At least if you are, it's not through intelligence, but it might be through access to more private information, or being rich enough to survive mistakes. As an example, you can't beat the stock market reliably with smarts, but you can by never being forced to sell. The real mystery to me is why people say "AI could recursively improve their own hardware and software in short time spans". I mean, that's clearly a made up concept since none of humans, computers or existing AI do it. But the closest thing I can think of is collective intelligence - humans individually haven't improved in the last 10k years, but we got a lot more humans and conquered everyone else that way. But we're also all individuals competing with each other and paying for our own individual food/maintenance/etc, which makes it different from nodes in an ever-growing AI.
hackerlight 4y ago
That's disanalogous, because: - children won't be smarter than any living human. - children have a human brain which makes them predictable (constrained in behavior by current laws, institutions, and most likely a conscience). A better analogy is to ask what happens when a species branches off and evolves into a smarter species, but the dumber ancestor species still exists. I really don't understand the perspective that people in your position take. Is it that you don't think we'll arrive at super intelligent AI, and therefore there's little risk? Or that we will be able to control it? If you think we can control it, why? Like we're not that much smarter than our monkey ancestors and what hope did they ever have of stopping our absolute domination over them? And then all of you call this opinion a "religion" without even explaining why it's wrong?
- mach1ne 4y ago
  Restricting AI to the lower end of human intelligence (e.g. around IQ of 70) makes it a useful resource which is guaranteed to be safe. A 70 IQ human couldn't take over the world nor disarm any safety features built into their bodies. You're right on the spot regarding the problem of having two different smart species on the same planet. We killed everything between us and chimps. Given enough time, the smarter species can be assumed to always take over.
  - deleted 4y ago
    [deleted]
  - astrange 4y ago
    That’s actually reverse causation - it’s not that we killed “everyone else”, it’s that everyone still alive became “us” after we gave up killing them and interbred with them instead. The British were going around genociding all over the place until pretty recently but now they’ve decided the Irish are human too. Homo sapiens is still expanding even: https://dna-explained.com/2012/11/16/the-new-root-haplogroup-a00/ https://dna-explained.com/2012/11/16/the-new-root-haplogroup...
    - hackerlight 4y ago

namlem 4y ago

AGI could be dangerous, but most people in AI safety are doing nothing useful to mitigate that risk.