11 ms·

While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?

brigadier132 2y ago
[flagged]
Dalewyn 2y ago
"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dangers" is a gross misrepresentation.
- comp_throw7 2y ago
  > * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. ...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."
  - Dalewyn 2y ago
    An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no, and I say that as a Japanese-American who visited Hiroshima Peace Memorial numerous times and never felt anything particular from the exhibits. People die when they are killed. Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings.
    - mlyle 2y ago
      > Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings. I think a dog and a dictator might both be able to have negative impacts, but treating them as equal just shows a desire to troll. A dog can only bite so many people.
    - comp_throw7 2y ago
      > People die when they are killed. Yes, indeed, which is why we should avoid creating something that will kill everyone... ...so I guess I'm glad to hear there's no difference in the amount of damage that can be inflicted by a baby, and by Vladimir Putin, and superintelligent AI really can't be any worse than either.
    - BuyMyBitcoins 2y ago
      >”An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no” Atomic bombs and hydrogen bombs are orders of magnitude more destructive and also produce fallout. How in the world can that difference be dismissed?
      - jiggawatts 2y ago
        Not the original commenter, but one interpretation is that “dead is dead”. You can’t be “more killed”, so past a point, all dangerous things are equally dangerous.
        Dalewyn 2y ago
        This. I don't see nuclear weapons as any worse than conventional weapons because ultimately people die. Is it comforting or otherwise somehow better for people to be killed by conventional weapons instead of nuclear weapons? I find such a proposition preposterous. All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options remain. Likewise, "AI" is just as much a threat as any other intelligent being or computer program depending on which line of thought you subscribe to. "AI" is not a "gravest of dangers".
        BuyMyBitcoins 2y ago
        >”I don't see nuclear weapons as any worse than conventional weapons because ultimately people die.” If you believe this, I am genuinely curious as to what you would consider the “gravest of dangers” to be.
        Dalewyn 2y ago
        I think of threats that are unknown and/or can't be adequately defended against: * Asteroid impacts like Chicxulub. * The Sun eventually inflating into a red giant and eating Earth, though this assumes we're both still around and haven't become spaceborne en masse. * Pandemics such as The Black Death and, indeed, covid for a recent example. * Social, political, or commercial intrigue. * Cancer. "AI" is a known threat that can be adequately defended against, so I agree with the sentiment that it's stupid to call it a "gravest of dangers".
    - brandall10 2y ago
      There absolutely is a difference. Nuclear war is considered a likely extinction level event for humanity. A war between the US/NATO and the Soviet Union could take out major cities around the globe within hours, and the resultant nuclear winter would blanket much of the planet within a month making living above ground impossible. GP's comparison is apt here.
      - 9dev 2y ago
        Just a minor correction, even though I agree with your point: the Soviet Union is no longer a thing, despite Putins wishes to the contrary :)
        TeMPOraL 2y ago
        Right, but its nukes are still around and pointed at NATO :).
    - komali2 2y ago
      Can you explain, with your worldview, why the nature of geopolitics turned abruptly and completely upon the invention and proliferation of nuclear bombs? What was it about the nuclear bomb that caused it to be the defining structure of the cold war? Do you believe MAD can exist without nuclear bombs?
      - Dalewyn 2y ago
        >What was it about the nuclear bomb that caused it to be the defining structure of the cold war? The only difference is you can kill more humans for a given span of time. The end result is the same: Cities are leveled, people are killed. Is it more comforting to have been killed by a conventional bomb than a nuke? I certainly don't think so. So practically speaking, there is no difference. >Do you believe MAD can exist without nuclear bombs? Yes. I think you're all vastly underestimating just how much destructive power conventionals can deliver already. You will need more ordnance on standby, but you absolutely can suffer from MAD without nukes.
        komali2 2y ago
        Do you believe any country has enough conventional ordinance to end human life on the entire planet? You must know more about this than me, I've never heard of something like this.
        Dalewyn 2y ago
        Any of the major military superpowers have more than enough conventional boomsticks to end all human life. It might take slightly longer than if nukes were used, but that's hardly comforting.
        comp_throw7 2y ago
        (For confused readers: no, they don't. Not even the nuclear stockpiles would be enough to cause human extinction, even if they were turned deliberately to that end.)
    - vouwfietsman 2y ago
      I would say at the very least that the empire of Japan clearly evaluated them differently when it surrendered unconditionally directly after being attacked with this new technology. You are saying magnitudes don't influence the identity of something, this is obviously not true. I don't literally attract other objects, the Sun does. The difference is mostly magnitude of mass.
      - Dalewyn 2y ago
        >I would say at the very least that the empire of Japan clearly evaluated them differently when it surrendered unconditionally directly after being attacked with this new technology. The atomic bombs certainly played a part in Japan surrendering, but it wasn't the only one. They were also facing a land invasion from the Soviets, had an utterly depleted military, waning support from the public, and a potentially devastating land invasion by the US on their doorstep. >You are saying magnitudes don't influence the identity of something, this is obviously not true. I am considering these things very dryly and very objectively. I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters. We already know how much damage an ill-willed intelligent being can achieve, we already know how much more efficient computers can make certain tasks. "AI" is not a "gravest of dangers", given that knowledge.
        TeMPOraL 2y ago
        > I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters. But what results are you really considering? The "dead is dead" approach makes everything equally meaningless, because in couple hundred million years the Sun will burn the Earth to cinders and everything here will be dead too. Or, some 100-120 years from now, everyone now alive will be dead, etc. Processes, and emotional pressures which steer those processes, matter too. In theory, machine guns could be used to kill everyone on the planet; in practice, it's economically and socially impossible - too much effort, too many people involved. Nuclear weapons, on the other hand, can realistically achieve this outcome. Which is why nukes can be used to keep the world mostly at peace through MAD, while conventional weapons can not.
- Jedd 2y ago
  I don't believe asserting two parallel and comparably false equivalences meaningfully progresses the discussion around this technology and risk. If this tech exceeds human capabilities to 'think' (plan, reason, manipulate, etc) then it is obviously a bigger threat than any less capable entity ('any living being', as you refer). Because no humans will be able to out-plan/reason/manipulate it. If the tech doesn't achieve intelligence - a tricky goal, and I eschew mislabelling LLMs as AIs for that reason - that doesn't mean its risk is comparable to the worst of any existing technology. There's just no way to extrapolate from one to the other, there. It might be orders of magnitude worse than any other 'computer program' - you can't say for sure, from where we're standing now.
  - Aexian 2y ago
    > you can't say for sure, from where we're standing now. We can already see how LLMs can be substantially worse - they can cosplay human thoughts and sentiment in a way that wasn’t previously possible. So much of this debate seems focused on this conception of Skynet-style murderous AI - or at least manipulative and scheming HAL 9000 types. But an order of magnitude greater risk has already arrived just from scaling existing harms. Phishing schemes and pig butchering scams have destroyed countless lives. They’re now easier and more scalable than ever. As are fake news and disinformation campaigns. Several companies are productising AI girlfriends with predatory pricing models, capitalising the human desire for connection and intimacy in the way slot machines and sports betting monetise our desire for a better life. That’s new. It may not be an order of magnitude worse for everyone yet - but for certain vulnerable groups, that future has arrived already.
- 0xDEAFBEAD 2y ago
  >* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.
  - blksv 2y ago
    But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. I agree, that AI may be dangerous if used for destructive purposes, or if used for some critical tasks with too much trust (and the hype that "we're so dangerously near a superintelligence" makes the latter much more likely, in my opinion). But that Humanity will be displaced by autonomous server farms? No way. (As to the original comment, I think that a bunch of nearly(?) demented elders holding nuclear buttons is a much worse (and immediate!) threat than a server farm which we finally conclude to be intelligent.)
    - 0xDEAFBEAD 2y ago
      >But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. One gorilla says to the other: "Those human brains are just synapses firing. They depend on nature to survive. Not a problem" Another way of thinking about it... Suppose we create a server emulation of a highly intelligent, manipulative serial killer, and speed up the emulation so it thinks 1000x as fast as a human. How do you feel about this? Is the fact that it's "just a server farm" reassuring?
      - blksv 2y ago
- EGreg 2y ago
  If "AI" is actually intelligent, then it's no worse a threat than any living being known to man How does the conclusion follow?
- edanm 2y ago
  > * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. That's not true, because you don't address the degree of intelligence or the scale that computers can achieve. Even if all we were able to do is to create human-intelligence-equivalent AIs, but we were able to create, say, a billion on command, I'd say we have a major threat.
  - d1sxeyes 2y ago
    One point to recall on this as well is that if we do develop an intelligent AI, it may have taken us millions of years, but we did it. An intelligent AI would potentially be able to iterate and self develop much faster than we have been able to. It probably won’t take long to go from ‘human equivalent’ to being beyond that.
    - edanm 2y ago
      That's true, but I long ago got convinced that this particular worry isn't the most important one (the recursive-self-improvement one). I'm fairly sure that Yudkowsky and others are also pushing that less, because there is a reasonable chance that it's just not a necessary part of the argument; even without self-improving, it's possible that just something like GPT-8 or something is harmful enough on its own that there are significant risks.
- ben_w 2y ago
  The most dangerous living beings in history have killed billions between them. Mostly that's diseases, but we can also easily add tens to hundreds of millions from humans fighting and starving each other over the last 200 years alone. Currently around a million a year killed in road traffic accidents, too, but I don't see stats for the total worldwide deaths since their invention. The greatest threats from other computer programs have been that during the Cold War the NATO early warning radar triggered an alert for a massive incoming fleet of Soviet aircraft because it hadn't been programmed to know that the moon wasn't supposed to respond to IFF pings, while the Soviet early warning radar triggered an alert an incoming ICBM because it hadn't been programmed to correctly handle the sun reflecting off clouds. It was human intervention which stopped that turning into a pointless self-annihilation, and that's specifically what gets lost as automation increases. I also do not believe that either of these threat models "are very thoroughly known and understood", because if the former was then we would've seen an end to genocides, while if the latter was we would've seen an end to zero-day vulnerabilities (especially in critical infrastructure).
yehabut 2y ago
That’s the thing though; reducing the risk rhetorically reduces existential threat. If it is propagated as “just electrons” no one gets anxious If it’s propagated as “end is nigh” everyone panics Both are reasonable in their grammatical correctness but only one seems a reasonable philosophy

[flagged]

"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dangers" is a gross misrepresentation.

comp_throw7 2y ago
> * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. ...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."
- Dalewyn 2y ago
  An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no, and I say that as a Japanese-American who visited Hiroshima Peace Memorial numerous times and never felt anything particular from the exhibits. People die when they are killed. Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings.
  - mlyle 2y ago
    > Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings. I think a dog and a dictator might both be able to have negative impacts, but treating them as equal just shows a desire to troll. A dog can only bite so many people.
  - comp_throw7 2y ago
    > People die when they are killed. Yes, indeed, which is why we should avoid creating something that will kill everyone... ...so I guess I'm glad to hear there's no difference in the amount of damage that can be inflicted by a baby, and by Vladimir Putin, and superintelligent AI really can't be any worse than either.
  - BuyMyBitcoins 2y ago
    >”An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no” Atomic bombs and hydrogen bombs are orders of magnitude more destructive and also produce fallout. How in the world can that difference be dismissed?
    - jiggawatts 2y ago
      Not the original commenter, but one interpretation is that “dead is dead”. You can’t be “more killed”, so past a point, all dangerous things are equally dangerous.
      - Dalewyn 2y ago
        This. I don't see nuclear weapons as any worse than conventional weapons because ultimately people die. Is it comforting or otherwise somehow better for people to be killed by conventional weapons instead of nuclear weapons? I find such a proposition preposterous. All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options remain. Likewise, "AI" is just as much a threat as any other intelligent being or computer program depending on which line of thought you subscribe to. "AI" is not a "gravest of dangers".
        BuyMyBitcoins 2y ago
        >”I don't see nuclear weapons as any worse than conventional weapons because ultimately people die.” If you believe this, I am genuinely curious as to what you would consider the “gravest of dangers” to be.
        Dalewyn 2y ago
        I think of threats that are unknown and/or can't be adequately defended against: * Asteroid impacts like Chicxulub. * The Sun eventually inflating into a red giant and eating Earth, though this assumes we're both still around and haven't become spaceborne en masse. * Pandemics such as The Black Death and, indeed, covid for a recent example. * Social, political, or commercial intrigue. * Cancer. "AI" is a known threat that can be adequately defended against, so I agree with the sentiment that it's stupid to call it a "gravest of dangers".
  - brandall10 2y ago
    There absolutely is a difference. Nuclear war is considered a likely extinction level event for humanity. A war between the US/NATO and the Soviet Union could take out major cities around the globe within hours, and the resultant nuclear winter would blanket much of the planet within a month making living above ground impossible. GP's comparison is apt here.
    - 9dev 2y ago
      Just a minor correction, even though I agree with your point: the Soviet Union is no longer a thing, despite Putins wishes to the contrary :)
      - TeMPOraL 2y ago
        Right, but its nukes are still around and pointed at NATO :).
  - komali2 2y ago
    Can you explain, with your worldview, why the nature of geopolitics turned abruptly and completely upon the invention and proliferation of nuclear bombs? What was it about the nuclear bomb that caused it to be the defining structure of the cold war? Do you believe MAD can exist without nuclear bombs?
    - Dalewyn 2y ago
      >What was it about the nuclear bomb that caused it to be the defining structure of the cold war? The only difference is you can kill more humans for a given span of time. The end result is the same: Cities are leveled, people are killed. Is it more comforting to have been killed by a conventional bomb than a nuke? I certainly don't think so. So practically speaking, there is no difference. >Do you believe MAD can exist without nuclear bombs? Yes. I think you're all vastly underestimating just how much destructive power conventionals can deliver already. You will need more ordnance on standby, but you absolutely can suffer from MAD without nukes.
      - komali2 2y ago
        Do you believe any country has enough conventional ordinance to end human life on the entire planet? You must know more about this than me, I've never heard of something like this.
        Dalewyn 2y ago
        Any of the major military superpowers have more than enough conventional boomsticks to end all human life. It might take slightly longer than if nukes were used, but that's hardly comforting.
        comp_throw7 2y ago
        (For confused readers: no, they don't. Not even the nuclear stockpiles would be enough to cause human extinction, even if they were turned deliberately to that end.)
  - vouwfietsman 2y ago
    I would say at the very least that the empire of Japan clearly evaluated them differently when it surrendered unconditionally directly after being attacked with this new technology. You are saying magnitudes don't influence the identity of something, this is obviously not true. I don't literally attract other objects, the Sun does. The difference is mostly magnitude of mass.
    - Dalewyn 2y ago
      >I would say at the very least that the empire of Japan clearly evaluated them differently when it surrendered unconditionally directly after being attacked with this new technology. The atomic bombs certainly played a part in Japan surrendering, but it wasn't the only one. They were also facing a land invasion from the Soviets, had an utterly depleted military, waning support from the public, and a potentially devastating land invasion by the US on their doorstep. >You are saying magnitudes don't influence the identity of something, this is obviously not true. I am considering these things very dryly and very objectively. I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters. We already know how much damage an ill-willed intelligent being can achieve, we already know how much more efficient computers can make certain tasks. "AI" is not a "gravest of dangers", given that knowledge.
      - TeMPOraL 2y ago
        > I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters. But what results are you really considering? The "dead is dead" approach makes everything equally meaningless, because in couple hundred million years the Sun will burn the Earth to cinders and everything here will be dead too. Or, some 100-120 years from now, everyone now alive will be dead, etc. Processes, and emotional pressures which steer those processes, matter too. In theory, machine guns could be used to kill everyone on the planet; in practice, it's economically and socially impossible - too much effort, too many people involved. Nuclear weapons, on the other hand, can realistically achieve this outcome. Which is why nukes can be used to keep the world mostly at peace through MAD, while conventional weapons can not.
Jedd 2y ago
I don't believe asserting two parallel and comparably false equivalences meaningfully progresses the discussion around this technology and risk. If this tech exceeds human capabilities to 'think' (plan, reason, manipulate, etc) then it is obviously a bigger threat than any less capable entity ('any living being', as you refer). Because no humans will be able to out-plan/reason/manipulate it. If the tech doesn't achieve intelligence - a tricky goal, and I eschew mislabelling LLMs as AIs for that reason - that doesn't mean its risk is comparable to the worst of any existing technology. There's just no way to extrapolate from one to the other, there. It might be orders of magnitude worse than any other 'computer program' - you can't say for sure, from where we're standing now.
- Aexian 2y ago
  > you can't say for sure, from where we're standing now. We can already see how LLMs can be substantially worse - they can cosplay human thoughts and sentiment in a way that wasn’t previously possible. So much of this debate seems focused on this conception of Skynet-style murderous AI - or at least manipulative and scheming HAL 9000 types. But an order of magnitude greater risk has already arrived just from scaling existing harms. Phishing schemes and pig butchering scams have destroyed countless lives. They’re now easier and more scalable than ever. As are fake news and disinformation campaigns. Several companies are productising AI girlfriends with predatory pricing models, capitalising the human desire for connection and intimacy in the way slot machines and sports betting monetise our desire for a better life. That’s new. It may not be an order of magnitude worse for everyone yet - but for certain vulnerable groups, that future has arrived already.
0xDEAFBEAD 2y ago
>* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.
- blksv 2y ago
  But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. I agree, that AI may be dangerous if used for destructive purposes, or if used for some critical tasks with too much trust (and the hype that "we're so dangerously near a superintelligence" makes the latter much more likely, in my opinion). But that Humanity will be displaced by autonomous server farms? No way. (As to the original comment, I think that a bunch of nearly(?) demented elders holding nuclear buttons is a much worse (and immediate!) threat than a server farm which we finally conclude to be intelligent.)
  - 0xDEAFBEAD 2y ago
    >But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. One gorilla says to the other: "Those human brains are just synapses firing. They depend on nature to survive. Not a problem" Another way of thinking about it... Suppose we create a server emulation of a highly intelligent, manipulative serial killer, and speed up the emulation so it thinks 1000x as fast as a human. How do you feel about this? Is the fact that it's "just a server farm" reassuring?
    - blksv 2y ago
EGreg 2y ago
If "AI" is actually intelligent, then it's no worse a threat than any living being known to man How does the conclusion follow?
edanm 2y ago
> * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. That's not true, because you don't address the degree of intelligence or the scale that computers can achieve. Even if all we were able to do is to create human-intelligence-equivalent AIs, but we were able to create, say, a billion on command, I'd say we have a major threat.
- d1sxeyes 2y ago
  One point to recall on this as well is that if we do develop an intelligent AI, it may have taken us millions of years, but we did it. An intelligent AI would potentially be able to iterate and self develop much faster than we have been able to. It probably won’t take long to go from ‘human equivalent’ to being beyond that.
  - edanm 2y ago
    That's true, but I long ago got convinced that this particular worry isn't the most important one (the recursive-self-improvement one). I'm fairly sure that Yudkowsky and others are also pushing that less, because there is a reasonable chance that it's just not a necessary part of the argument; even without self-improving, it's possible that just something like GPT-8 or something is harmful enough on its own that there are significant risks.
ben_w 2y ago
The most dangerous living beings in history have killed billions between them. Mostly that's diseases, but we can also easily add tens to hundreds of millions from humans fighting and starving each other over the last 200 years alone. Currently around a million a year killed in road traffic accidents, too, but I don't see stats for the total worldwide deaths since their invention. The greatest threats from other computer programs have been that during the Cold War the NATO early warning radar triggered an alert for a massive incoming fleet of Soviet aircraft because it hadn't been programmed to know that the moon wasn't supposed to respond to IFF pings, while the Soviet early warning radar triggered an alert an incoming ICBM because it hadn't been programmed to correctly handle the sun reflecting off clouds. It was human intervention which stopped that turning into a pointless self-annihilation, and that's specifically what gets lost as automation increases. I also do not believe that either of these threat models "are very thoroughly known and understood", because if the former was then we would've seen an end to genocides, while if the latter was we would've seen an end to zero-day vulnerabilities (especially in critical infrastructure).

yehabut 2y ago

That’s the thing though; reducing the risk rhetorically reduces existential threat. If it is propagated as “just electrons” no one gets anxious If it’s propagated as “end is nigh” everyone panics Both are reasonable in their grammatical correctness but only one seems a reasonable philosophy