4 ms·

>> Big asteroid impact (historically unlikely) Not sure if I would call this historically unlikely, definitely extremely infrequent but almost guaranteed to ha

by pr0zac 3y ago

>> Big asteroid impact (historically unlikely) Not sure if I would call this historically unlikely, definitely extremely infrequent but almost guaranteed to happen again eventually, just maybe on a very large time scale. On the other hand, I think its quite possible some small amount of humans would survive a Chicxulub size asteroid. I'd probably classify its potential damage to the species similarly to other mass starvation causing environmental disasters like climate change.

myrmidon 3y ago
I think it would be fair to give the asteroid impact roughly a one in a million odds over a human lifespan-- speculating over longer intervals is pointless anyway because we already have some countermeasures and any impactor smaller than the moon might become divertible within a century or two. On the other hand, it seems VERY likely to me that an AI with superhuman cognitive capabilities is just a matter of time (lets say a decade or two), and the risk that such a system would NOT favor coexistence with humans appears MUCH more likely than 1/1000000 to me: Just consider the current debate about "alignment", which to me just seems obviously and inherently unethical; I would not want to coexist with a species whose "ethical" concerns are largely centered around controlling and manipulating me reliably.
- jazzyjackson 3y ago
  can you tell me why it seems likely to you? obvious, inevitable, these are religious ways to reason. what coexistence are we talking about beyond how we co-exist with existing software?
  - myrmidon 3y ago
    I assume we're talking about why I deem it "likely" (lets say the likelihood is 1%-- thats already 10000 times more threathening than the asteroid) that an AI might not want to coexist with us: This is because it appears clear to me that it would be unlikely and hard for an AI to receive "fair" treatment, i.e. rights comparable to a human being (given comparable cognitive capabilities). The current focus of AI ethics supports this view ("aligning" the AIs decisionmaking with what humans want from it: this would be called indoctrination or brainwashing with humans, and generally not viewed favorably), as do instances where biological humans are very clearly favored over entities with comparable cognitive capabilities: Compare legal rights of retarded humans with those of cattle (sorry for crass tone, not ment offensively).

by pr0zac 3y ago

myrmidon 3y ago

I think it would be fair to give the asteroid impact roughly a one in a million odds over a human lifespan-- speculating over longer intervals is pointless anyway because we already have some countermeasures and any impactor smaller than the moon might become divertible within a century or two. On the other hand, it seems VERY likely to me that an AI with superhuman cognitive capabilities is just a matter of time (lets say a decade or two), and the risk that such a system would NOT favor coexistence with humans appears MUCH more likely than 1/1000000 to me: Just consider the current debate about "alignment", which to me just seems obviously and inherently unethical; I would not want to coexist with a species whose "ethical" concerns are largely centered around controlling and manipulating me reliably.

jazzyjackson 3y ago
can you tell me why it seems likely to you? obvious, inevitable, these are religious ways to reason. what coexistence are we talking about beyond how we co-exist with existing software?
- myrmidon 3y ago
  I assume we're talking about why I deem it "likely" (lets say the likelihood is 1%-- thats already 10000 times more threathening than the asteroid) that an AI might not want to coexist with us: This is because it appears clear to me that it would be unlikely and hard for an AI to receive "fair" treatment, i.e. rights comparable to a human being (given comparable cognitive capabilities). The current focus of AI ethics supports this view ("aligning" the AIs decisionmaking with what humans want from it: this would be called indoctrination or brainwashing with humans, and generally not viewed favorably), as do instances where biological humans are very clearly favored over entities with comparable cognitive capabilities: Compare legal rights of retarded humans with those of cattle (sorry for crass tone, not ment offensively).