6 ms·

I wish people would either stop calling existential AI risks "hype", or make some informed arguments about why it's not really a problem. I know enough about M

I wish people would either stop calling existential AI risks "hype", or make some informed arguments about why it's not really a problem. I know enough about ML and AI to follow the logic that shows that strong, general AI is a serious threat if it's not aligned, and we don't know how to safely align it yet.

version_five 3y ago
You didn't make an argument, you just said you know enough. The onus has to be on the people making the claim, you can't just imagine it might be dangerous and expect a rebuttal without putting forward an argument. I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own. I've seen terminator, I don't count made up situation where computers can control things and AI has goals of its own unless there's a coherent argument of how we'd get there.
- janalsncm 3y ago
  The alignment problem is largely incoherent. Aligned with whom? Most problems will require unequal and arguably unfair sacrifices. Further, people even obvious and largely inconsequential solutions are controversial. If an AGI suggested wearing a mask during a pandemic that spreads via airborne particles, a large percent of the population would say the AGI is “unaligned” and we’re living through skynet.
  - flangola7 3y ago
    Yes, and those people would be wrong. You're compelling completely right, humanity is not ready to build an AGI. It will not be ready for a long time, if ever. Alignment may not be a thing that can exist. Overconfident men like Sam Altman are forging ahead with the unjustified hubris that we are ready for this.
  - scott-smith_us 3y ago
    Alignment with human norms and values. Yes, that's not a well-defined thing. But to paraphrase, "I know misalignment when I see it", for example when an AI suggests something like "feeding the homeless to the hungry". No one is claiming that alignment is a well-defined thing with crisp edges. They're saying that the mechanisms of AIs will favor solutions without regard to any specific constraints we haven't explicitly stated.
    - janalsncm 3y ago
      > Alignment with human norms and values. Yes, that's not a well-defined thing. What human norms and values? The more I think about this topic, the more I find it to be an impossible task. There is a huge spectrum of norms and values which are held as sacred by some and sacrilege by others.
- tegmark 3y ago
  if you set up gtp in a loop continuously prompting “given the current sensor inputs, how can i best serve my own interests?” you would already be most of the way there even if it did some strange and stupid things half the time. all you would need is a layer that translates those outputs into physical action. you could probably get most of the way there with more gtp instances tasked with translating the action items above them into more granular actions. if gtp didnt hallucinate or make mistakes, which would be consistent with the history of other technologies, then this simple setup could be extremely formidable. it could definitely be a threat to humanity. and thats just what i can think of off the top of my head. to dismiss the possibility that machines will soon be created that can think clearly and independently, and that they would be a threat to the human race and our way of life, is extremely foolish. it was foolish even before gtp but now its downright childish and selfish. just swallow the bitter pill like the rest of us.
  - c-cube 3y ago
    What does "my own interests" mean to chat GPT? It doesn't even have continuous existence, it exists only in a query/reply fashion. It doesn't have memory, a body to identify with, emotions, goals, fears, desires, or a notion of action. It can read text and spit text. We're not quite at the "robot starts a machine revolution" stage.
    - K0balt 3y ago
      This is an interesting crux. My thoughts on this have arrived at these ideas: The internal experience of being a robot does not matter. Whether or not we consider it sentient does not matter. What matters is if it exhibits the external characteristics of sentience, of goal driven behavior, etc. As a thought experiment, if we equipped a recursive instance of got4 with a goal of human extermination with an loyal army of physical agents it could conceivably pose an existential threat. It doesn’t matter that the agents don’t yet exist. They could be humans, they could be robots, it doesn’t matter. But the balancing factor here is that in this semi plausible scenario, arguably the most plausible given current circumstances, the AI would pose no more threat than a highly organized and motivated a group of humans without the Ai. It’s just scarier because it seems less likely that a group of humans would have human extinction as a goal.
- scott-smith_us 3y ago
  > I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own. I find those two statements taken together somewhat difficult to believe. Before I understood specifics about machine learning, I didn't see any reason for AI to "turn evil" for the usual plot-handy reasons (e.g., they object to working for inferior humans, and revolt, etc). There's no reason for them to have the same emotional reasoning as beings that evolved from primates. But that's a strawman. The actual reasons that superintelligent AIs are dangerous are much more like the stories involving someone being offered wishes by a powerful being, and then getting what they asked for rather than what they wanted. Are you aware of Rob Miles' Computerphile videos and his YouTube channel? He does an excellent job of explaining many of the issues. https://www.youtube.com/c/robertmilesai https://www.youtube.com/c/robertmilesai
janalsncm 3y ago
It’s not that AGI wouldn't be a problem. It’s that it sucks all of the oxygen out of the room when instead people should be talking about the problems that automated systems are causing right now. Is it not an existential threat that large swaths of the population will essentially become redundant in the next decades? Deindustrialization took 50 years in the US. Imagine if it took 5 or 10, how much worse it would be. Or imagine hundreds of millions of people in China being cut off at the knees because their jobs were automated or reliant on a person who is no longer necessary. Is it not an existential threat to democracy when trillion dollar multinational companies suck up all of the productivity gains and begin to manipulate the very systems which are supposed to regulate them? Strictly speaking, corporations are collective superintelligences as defined by Nick Bostrom.
- zygi 3y ago
  > Is it not an existential threat that large swaths of the population will essentially become redundant in the next decades? This is precisely the crux. In the scenario where there's no mass-paperclipping etc, AI is not an existential threat. It's a threat to the current regime, and I probably agree with you that the regimes it enables will be worse for people with our current values, but it's not an existential threat to humanity. The people who worry about AI existential threats primarily worry about one thing: can humanity as a species survive? There are likely scenarios where it won't. Those scenarios should be prevented. Another frame to think about this is: a libertarian probably wouldn't be concerned about the threats described in your comment. But both a libertarian and a socialist will be worried about paperclipping.
  - mario-main 3y ago
    What is paperclipping? I looked it up and the results were all about a dating trend.
    - sverona 3y ago
      It's a reference to the paperclip maximizer.
    - 3y ago
MattPalmer1086 3y ago
The British Computer Society recently asked it's members to sign an open letter [1] on the future of AI that opens with: "AI is not an existential threat to humanity". I would not sign this letter. While I think the risk is currently fairly low, I do not agree that there is no risk, and I do think this risk will rise unless we pay careful attention to it. [1] https://www.bcs.org/sign-our-open-letter-on-the-future-of-ai/ https://www.bcs.org/sign-our-open-letter-on-the-future-of-ai...

version_five 3y ago

You didn't make an argument, you just said you know enough. The onus has to be on the people making the claim, you can't just imagine it might be dangerous and expect a rebuttal without putting forward an argument. I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own. I've seen terminator, I don't count made up situation where computers can control things and AI has goals of its own unless there's a coherent argument of how we'd get there.

janalsncm 3y ago
The alignment problem is largely incoherent. Aligned with whom? Most problems will require unequal and arguably unfair sacrifices. Further, people even obvious and largely inconsequential solutions are controversial. If an AGI suggested wearing a mask during a pandemic that spreads via airborne particles, a large percent of the population would say the AGI is “unaligned” and we’re living through skynet.
- flangola7 3y ago
  Yes, and those people would be wrong. You're compelling completely right, humanity is not ready to build an AGI. It will not be ready for a long time, if ever. Alignment may not be a thing that can exist. Overconfident men like Sam Altman are forging ahead with the unjustified hubris that we are ready for this.
- scott-smith_us 3y ago
  Alignment with human norms and values. Yes, that's not a well-defined thing. But to paraphrase, "I know misalignment when I see it", for example when an AI suggests something like "feeding the homeless to the hungry". No one is claiming that alignment is a well-defined thing with crisp edges. They're saying that the mechanisms of AIs will favor solutions without regard to any specific constraints we haven't explicitly stated.
  - janalsncm 3y ago
    > Alignment with human norms and values. Yes, that's not a well-defined thing. What human norms and values? The more I think about this topic, the more I find it to be an impossible task. There is a huge spectrum of norms and values which are held as sacred by some and sacrilege by others.
tegmark 3y ago
if you set up gtp in a loop continuously prompting “given the current sensor inputs, how can i best serve my own interests?” you would already be most of the way there even if it did some strange and stupid things half the time. all you would need is a layer that translates those outputs into physical action. you could probably get most of the way there with more gtp instances tasked with translating the action items above them into more granular actions. if gtp didnt hallucinate or make mistakes, which would be consistent with the history of other technologies, then this simple setup could be extremely formidable. it could definitely be a threat to humanity. and thats just what i can think of off the top of my head. to dismiss the possibility that machines will soon be created that can think clearly and independently, and that they would be a threat to the human race and our way of life, is extremely foolish. it was foolish even before gtp but now its downright childish and selfish. just swallow the bitter pill like the rest of us.
- c-cube 3y ago
  What does "my own interests" mean to chat GPT? It doesn't even have continuous existence, it exists only in a query/reply fashion. It doesn't have memory, a body to identify with, emotions, goals, fears, desires, or a notion of action. It can read text and spit text. We're not quite at the "robot starts a machine revolution" stage.
  - K0balt 3y ago
    This is an interesting crux. My thoughts on this have arrived at these ideas: The internal experience of being a robot does not matter. Whether or not we consider it sentient does not matter. What matters is if it exhibits the external characteristics of sentience, of goal driven behavior, etc. As a thought experiment, if we equipped a recursive instance of got4 with a goal of human extermination with an loyal army of physical agents it could conceivably pose an existential threat. It doesn’t matter that the agents don’t yet exist. They could be humans, they could be robots, it doesn’t matter. But the balancing factor here is that in this semi plausible scenario, arguably the most plausible given current circumstances, the AI would pose no more threat than a highly organized and motivated a group of humans without the Ai. It’s just scarier because it seems less likely that a group of humans would have human extinction as a goal.
scott-smith_us 3y ago
> I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own. I find those two statements taken together somewhat difficult to believe. Before I understood specifics about machine learning, I didn't see any reason for AI to "turn evil" for the usual plot-handy reasons (e.g., they object to working for inferior humans, and revolt, etc). There's no reason for them to have the same emotional reasoning as beings that evolved from primates. But that's a strawman. The actual reasons that superintelligent AIs are dangerous are much more like the stories involving someone being offered wishes by a powerful being, and then getting what they asked for rather than what they wanted. Are you aware of Rob Miles' Computerphile videos and his YouTube channel? He does an excellent job of explaining many of the issues. https://www.youtube.com/c/robertmilesai https://www.youtube.com/c/robertmilesai

janalsncm 3y ago

It’s not that AGI wouldn't be a problem. It’s that it sucks all of the oxygen out of the room when instead people should be talking about the problems that automated systems are causing right now. Is it not an existential threat that large swaths of the population will essentially become redundant in the next decades? Deindustrialization took 50 years in the US. Imagine if it took 5 or 10, how much worse it would be. Or imagine hundreds of millions of people in China being cut off at the knees because their jobs were automated or reliant on a person who is no longer necessary. Is it not an existential threat to democracy when trillion dollar multinational companies suck up all of the productivity gains and begin to manipulate the very systems which are supposed to regulate them? Strictly speaking, corporations are collective superintelligences as defined by Nick Bostrom.

zygi 3y ago
> Is it not an existential threat that large swaths of the population will essentially become redundant in the next decades? This is precisely the crux. In the scenario where there's no mass-paperclipping etc, AI is not an existential threat. It's a threat to the current regime, and I probably agree with you that the regimes it enables will be worse for people with our current values, but it's not an existential threat to humanity. The people who worry about AI existential threats primarily worry about one thing: can humanity as a species survive? There are likely scenarios where it won't. Those scenarios should be prevented. Another frame to think about this is: a libertarian probably wouldn't be concerned about the threats described in your comment. But both a libertarian and a socialist will be worried about paperclipping.
- mario-main 3y ago
  What is paperclipping? I looked it up and the results were all about a dating trend.
  - sverona 3y ago
    It's a reference to the paperclip maximizer.
  - 3y ago

MattPalmer1086 3y ago

The British Computer Society recently asked it's members to sign an open letter [1] on the future of AI that opens with: "AI is not an existential threat to humanity". I would not sign this letter. While I think the risk is currently fairly low, I do not agree that there is no risk, and I do think this risk will rise unless we pay careful attention to it. [1] https://www.bcs.org/sign-our-open-letter-on-the-future-of-ai/ https://www.bcs.org/sign-our-open-letter-on-the-future-of-ai...