5 ms·

This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provi

This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provide evidence that this is a credible threat in the real world.

jefftk 3y ago
The two people I cited at the end are examples of philosophers who think a lifeless world is preferable to the status quo and to futures we are likely to get. These aren't arguments for why someone might hold the belief, but evidence that some people do seriously hold the belief. (Another example here would be https://en.m.wikipedia.org/wiki/Aum_Shinrikyo https://en.m.wikipedia.org/wiki/Aum_Shinrikyo)
- jebarker 3y ago
  Fair enough. I don't doubt there are people in the world that hold this view. I just don't know how credible they are as a threat to humanity.
SkyMarshal 3y ago
Elon Musk and Joe Rogan were just complaining about one of them on Rogan’s podcast a day or two ago. He was on the cover of Time magazine, but I can’t find it on Time’s site. If you dig up that podcast on YT you should be able to see it.
pixl97 3y ago
The calculation of considering something is a credible threat directly corresponds with the ease of implementation. If I said "I am going to drop a nuke on jebarker" that's not a credible threat unless I happen to be a nation state. Now, if I said "I'm going to come to jebarkers house and blast him with a 12 gauge" and I'm from the US, that is a credible threat in most cases due to the ease of which I can get a weapon here. And this comes down to the point of having a magic machine in which you could ask for the end of the world. The more powerful that machine gets the bigger risk it is for everyone. You ever see those people that snap and start killing everyone around them? Would you want them to have a nuke in that mood? Or a 'lets turn humanity in to grey goo' option?
- jebarker 3y ago
  The problem with this analysis is that we have complete uncertainty about the probability that AGI/ASI will be developed in any particular time frame. Anyone that says otherwise is lying or deluded. So the risk equation is impossible to calculate for AGI/ASI wiping out humanity. And since you appear to be evaluating the damage as essentially infinite, i.e. existential risk, you're advocating that as long as there's a greater than zero probability of someone using that capability then the risk is infinite. Which is not useful for deciding a course of action.
  - pixl97 3y ago
    No, no need to calculate infinities. Instead lets imagine something with an IQ within one exponent higher of ours. This is exceptionally difficult to even imagine as below 70 we consider a person disabled in some form or another. Once we get in the range of IQ over 170 or so it really starts to break. When we get into multimodal self learning systems there may be vast amounts of knowledge encoded into its networks that we simply don't know the right question to ask as humans. Attempt to think of what other AIs at or near that level could ask each other. Our different systems of intelligence keep falling back to ideas of human intelligence and its limitations. Things like we don't multitask well, our best attention is given to one thing at a time. Things like our highest bandwidth senses are our ears and eyes, and even then they are considerably bandwidth constricted. Human limits like our abilities cannot be horizontally scaled in an easily measurable way, and adding more people massively increases the networking costs of accomplishing a goal. If for some reason the smartest AI can only reach the levels of the smartest human beings (which at least to me makes no sense), then this is still massively powerful, as of this point pesky human problems like 8 hours of sleep are not needed. If human level AI power can be shrunk down to near cellphone size and power consumption, which again doesn't seem out of the realm of physics, lays down the framework for an intelligence explosion at the rate which we can print chips out and assemble them.
    - jebarker 3y ago
      But the problem still stands that we can't estimate how likely any of this is to occur. So in the overall risk equation we have a term that is essentially completely uncertain and therefore varying that term can yield any arbitrary risk level you want.

jefftk 3y ago

The two people I cited at the end are examples of philosophers who think a lifeless world is preferable to the status quo and to futures we are likely to get. These aren't arguments for why someone might hold the belief, but evidence that some people do seriously hold the belief. (Another example here would be https://en.m.wikipedia.org/wiki/Aum_Shinrikyo https://en.m.wikipedia.org/wiki/Aum_Shinrikyo)

jebarker 3y ago
Fair enough. I don't doubt there are people in the world that hold this view. I just don't know how credible they are as a threat to humanity.

SkyMarshal 3y ago

Elon Musk and Joe Rogan were just complaining about one of them on Rogan’s podcast a day or two ago. He was on the cover of Time magazine, but I can’t find it on Time’s site. If you dig up that podcast on YT you should be able to see it.

pixl97 3y ago

The calculation of considering something is a credible threat directly corresponds with the ease of implementation. If I said "I am going to drop a nuke on jebarker" that's not a credible threat unless I happen to be a nation state. Now, if I said "I'm going to come to jebarkers house and blast him with a 12 gauge" and I'm from the US, that is a credible threat in most cases due to the ease of which I can get a weapon here. And this comes down to the point of having a magic machine in which you could ask for the end of the world. The more powerful that machine gets the bigger risk it is for everyone. You ever see those people that snap and start killing everyone around them? Would you want them to have a nuke in that mood? Or a 'lets turn humanity in to grey goo' option?

jebarker 3y ago
The problem with this analysis is that we have complete uncertainty about the probability that AGI/ASI will be developed in any particular time frame. Anyone that says otherwise is lying or deluded. So the risk equation is impossible to calculate for AGI/ASI wiping out humanity. And since you appear to be evaluating the damage as essentially infinite, i.e. existential risk, you're advocating that as long as there's a greater than zero probability of someone using that capability then the risk is infinite. Which is not useful for deciding a course of action.
- pixl97 3y ago
  No, no need to calculate infinities. Instead lets imagine something with an IQ within one exponent higher of ours. This is exceptionally difficult to even imagine as below 70 we consider a person disabled in some form or another. Once we get in the range of IQ over 170 or so it really starts to break. When we get into multimodal self learning systems there may be vast amounts of knowledge encoded into its networks that we simply don't know the right question to ask as humans. Attempt to think of what other AIs at or near that level could ask each other. Our different systems of intelligence keep falling back to ideas of human intelligence and its limitations. Things like we don't multitask well, our best attention is given to one thing at a time. Things like our highest bandwidth senses are our ears and eyes, and even then they are considerably bandwidth constricted. Human limits like our abilities cannot be horizontally scaled in an easily measurable way, and adding more people massively increases the networking costs of accomplishing a goal. If for some reason the smartest AI can only reach the levels of the smartest human beings (which at least to me makes no sense), then this is still massively powerful, as of this point pesky human problems like 8 hours of sleep are not needed. If human level AI power can be shrunk down to near cellphone size and power consumption, which again doesn't seem out of the realm of physics, lays down the framework for an intelligence explosion at the rate which we can print chips out and assemble them.
  - jebarker 3y ago
    But the problem still stands that we can't estimate how likely any of this is to occur. So in the overall risk equation we have a term that is essentially completely uncertain and therefore varying that term can yield any arbitrary risk level you want.