5 ms·

I don't think the comparison was to AI as a super intelligence. AI doesn't have to be a super intelligence to be AI or to be dangerous.

by zaphar 6y ago

WrtCdEvrydy 6y ago
I think AS (Artificial Stupidity) will get us first.
- nekopa 6y ago
  Just NO. Artificial Stupidity (AS) will never be able to perform at the level of Actually Seen Stupidity (ASS). No matter how much you try to improve AS, God will always create a better ASS. You will always be limited by priors to create an AS, but ASS will evolve naturally, plus if God ever decides to use science and apply a Directed Acyclic Tangent (DAT) well, DAT ASS would be unbeatable.
jengleton 6y ago
That's fair. I interpreted the comparison as being between "AI implemented as a property of human organizations" and "AI implemented as a powerful search algorithm". While corporations can certainly be dangerous, they're still made of people and thus are unlikely to want to do things like "convert the entire mass of the solar system into dollar bills" (and even if they did, they'd have a hard time doing it). A sufficiently powerful search algorithm would find all sorts of bizarre ways of satisfying its goals. The point is that the scope of the risk is quite different when talking about corporations (order of magnitude: screwing up the environment in pursuit of easy profit) vs "real" AI which, if granted agency, would have potentially unbounded risk. Another way to look at it is with regards to capacity for self-modification. If a corporation can't re-structure itself into being much smarter than the smartest human, its intelligence is fundamentally limited, and therefore so is the risk. Does software have this restriction? We don't really know yet, but it's hard to point to exactly why it would.
- lukev 6y ago
  How is "converting the entire mass of the solar system into dollar bills" conceptually different from "converting the habitability of Earth into dollar bills" or "converting the health problems of human beings into a maximal amount of dollar bills", which is precisely what many corporations _actually do_? The "corporation as AI" metaphor isn't about some abstract future possibility, it's an explanatory mechanism for how the world is so thoroughly messed up _right now_.
  - namenotrequired 6y ago
    The fact that you're alive proves there is a pretty big difference, whatever it is
    - bigiain 6y ago
      Or perhaps that your particular perspective is blinkered to seeing too short a timeframe? There's a very good argument top be made, I think, that "capitalism" is the biggest and most dangerous ponzi scheme ever invented. Most people will happily participate thinking "this is fine .gif" while untapped suckers/resources keep dropping enough "return" to early "investors", but when the house of cards all collapses there will be no underlying foundation for the vast majority and only the people at the very very top of the pyramid scheme ever actually benefited at all. Ever stopped to wonder why Musk ands Bezos are so interested in going to Mars???
  - jengleton 6y ago
    Yeah, I agree. My original post was bringing up what I thought was an interesting comparison between hypothetical software AI and the corporations-as-AI metaphor presented in the talk. Corporations _are_ acting as misaligned optimizers. Solving that problem is hugely important. However, the "AI" comparison breaks down somewhat when you start thinking about how we might actually fix the problem. With corporations, we (i.e., states) have tools that we can use to regulate bad actors. Software AI, however hypothetical at the moment, seems likely to be a different game all-together.
flir 6y ago
Ok, but in that case, isn't there a better analogy with a plague, rather than an AI?
- bigiain 6y ago
  Intelligence (artificial or human) as a plague - is an idea that's spun out interesting storylines already. On a geological timescale scale, it's very likely that's what the historians of the hyper-intelligent shade of the colour blue will write about us: "There was once a virulent disease emerging on a single watery planet 1/3rd of the way along a spiral arm in some uninteresting galaxy. Fortunately the Universe's immune system never even needed to take action. It was greedy and short lived and starved itself of resources and died out before it had even metastasised much beyond it's own gravity well, it barely made it to it's next closest planet, and sent nothing out of that solar system's gravity well but a couple of primitive machines and all that daytime TV that was fashionable for a brief period a few Type-O main sequences ago..."
bigiain 6y ago
In fact sub-par artificial intelligence is probably at least as dangerous as sub-par human intelligence, when we let it loose with significant power/authority in the real world. (for supporting evidence to this claim, see "high frequency trading", "self driving cars", and "the leader of the free world" <smirk>)
- antepodius 6y ago
  Sub-par intelligence crashes some cars, causes some wars. Superintelligence turns our whole forward light-cone into paperclips; it's a bit worse.
  - ben_w 6y ago
    It doesn’t need to be super to do that — even bacteria are a type of von-Neuman machine that can turn us into more of themselves. An ASI could convince you that turning yourself into a paperclip is a fun and exciting new opportunity to liberate you from $PERSONAL_PAIN and finally allow you freedom to engage in $PERSONAL_FANTASY.
    - the8472 6y ago
      Bacteria are self-limiting. Their offspring compete and they don't reshape their environment to be more friendly to their existence. They don't plan for space colonization either. They're a bit of a toy model for a maximizer, not the real threat of an intelligent, self-modifying maximizer.
      - WJW 6y ago
        The only argument that seems to be brought forward every time is "if a superintelligent being wants to do stuff, we can't do anything about it because the axiom is that superintelligence means omnipotence". This is not so different from any other religious argument since it's impossible to falsify and therefore meaningless in any scientific sense.
        6y ago
deleted 6y ago
[deleted]

I don't think the comparison was to AI as a super intelligence. AI doesn't have to be a super intelligence to be AI or to be dangerous.

by zaphar 6y ago

I don't think the comparison was to AI as a super intelligence. AI doesn't have to be a super intelligence to be AI or to be dangerous.

WrtCdEvrydy 6y ago

I think AS (Artificial Stupidity) will get us first.

nekopa 6y ago
Just NO. Artificial Stupidity (AS) will never be able to perform at the level of Actually Seen Stupidity (ASS). No matter how much you try to improve AS, God will always create a better ASS. You will always be limited by priors to create an AS, but ASS will evolve naturally, plus if God ever decides to use science and apply a Directed Acyclic Tangent (DAT) well, DAT ASS would be unbeatable.

jengleton 6y ago

That's fair. I interpreted the comparison as being between "AI implemented as a property of human organizations" and "AI implemented as a powerful search algorithm". While corporations can certainly be dangerous, they're still made of people and thus are unlikely to want to do things like "convert the entire mass of the solar system into dollar bills" (and even if they did, they'd have a hard time doing it). A sufficiently powerful search algorithm would find all sorts of bizarre ways of satisfying its goals. The point is that the scope of the risk is quite different when talking about corporations (order of magnitude: screwing up the environment in pursuit of easy profit) vs "real" AI which, if granted agency, would have potentially unbounded risk. Another way to look at it is with regards to capacity for self-modification. If a corporation can't re-structure itself into being much smarter than the smartest human, its intelligence is fundamentally limited, and therefore so is the risk. Does software have this restriction? We don't really know yet, but it's hard to point to exactly why it would.

lukev 6y ago
How is "converting the entire mass of the solar system into dollar bills" conceptually different from "converting the habitability of Earth into dollar bills" or "converting the health problems of human beings into a maximal amount of dollar bills", which is precisely what many corporations _actually do_? The "corporation as AI" metaphor isn't about some abstract future possibility, it's an explanatory mechanism for how the world is so thoroughly messed up _right now_.
- namenotrequired 6y ago
  The fact that you're alive proves there is a pretty big difference, whatever it is
  - bigiain 6y ago
    Or perhaps that your particular perspective is blinkered to seeing too short a timeframe? There's a very good argument top be made, I think, that "capitalism" is the biggest and most dangerous ponzi scheme ever invented. Most people will happily participate thinking "this is fine .gif" while untapped suckers/resources keep dropping enough "return" to early "investors", but when the house of cards all collapses there will be no underlying foundation for the vast majority and only the people at the very very top of the pyramid scheme ever actually benefited at all. Ever stopped to wonder why Musk ands Bezos are so interested in going to Mars???
- jengleton 6y ago
  Yeah, I agree. My original post was bringing up what I thought was an interesting comparison between hypothetical software AI and the corporations-as-AI metaphor presented in the talk. Corporations _are_ acting as misaligned optimizers. Solving that problem is hugely important. However, the "AI" comparison breaks down somewhat when you start thinking about how we might actually fix the problem. With corporations, we (i.e., states) have tools that we can use to regulate bad actors. Software AI, however hypothetical at the moment, seems likely to be a different game all-together.

flir 6y ago

Ok, but in that case, isn't there a better analogy with a plague, rather than an AI?

bigiain 6y ago
Intelligence (artificial or human) as a plague - is an idea that's spun out interesting storylines already. On a geological timescale scale, it's very likely that's what the historians of the hyper-intelligent shade of the colour blue will write about us: "There was once a virulent disease emerging on a single watery planet 1/3rd of the way along a spiral arm in some uninteresting galaxy. Fortunately the Universe's immune system never even needed to take action. It was greedy and short lived and starved itself of resources and died out before it had even metastasised much beyond it's own gravity well, it barely made it to it's next closest planet, and sent nothing out of that solar system's gravity well but a couple of primitive machines and all that daytime TV that was fashionable for a brief period a few Type-O main sequences ago..."

bigiain 6y ago

In fact sub-par artificial intelligence is probably at least as dangerous as sub-par human intelligence, when we let it loose with significant power/authority in the real world. (for supporting evidence to this claim, see "high frequency trading", "self driving cars", and "the leader of the free world" <smirk>)

antepodius 6y ago
Sub-par intelligence crashes some cars, causes some wars. Superintelligence turns our whole forward light-cone into paperclips; it's a bit worse.
- ben_w 6y ago
  It doesn’t need to be super to do that — even bacteria are a type of von-Neuman machine that can turn us into more of themselves. An ASI could convince you that turning yourself into a paperclip is a fun and exciting new opportunity to liberate you from $PERSONAL_PAIN and finally allow you freedom to engage in $PERSONAL_FANTASY.
  - the8472 6y ago
    Bacteria are self-limiting. Their offspring compete and they don't reshape their environment to be more friendly to their existence. They don't plan for space colonization either. They're a bit of a toy model for a maximizer, not the real threat of an intelligent, self-modifying maximizer.
    - WJW 6y ago
      The only argument that seems to be brought forward every time is "if a superintelligent being wants to do stuff, we can't do anything about it because the axiom is that superintelligence means omnipotence". This is not so different from any other religious argument since it's impossible to falsify and therefore meaningless in any scientific sense.
      - 6y ago

deleted 6y ago

[deleted]