6 ms·
It is. And it gets clicks. But what I rarely see discussed is the opportunity costs in not having this progress at as fast a pace as possible. The pie chart o
by kromem 3y ago
It is. And it gets clicks.
But what I rarely see discussed is the opportunity costs in not having this progress at as fast a pace as possible.
The pie chart of existential threats for humanity definitely has rouge AI on it.
But that's a slice amidst many human driven threats ranging from nuclear war to oceans dying.
What there's not very many human driven slices of pie for is realistic solutions to these issues.
On that pie chart, some sort of AGI deus ex machina is at the current rate of progress probably the only realistic hope we have.
People who have been in the AI industry for a long time are still operating on thought experiments that barely resemble the actual emergence we are watching.
So while you have old AI thought leaders going on about cold heartless AI not being able to stop itself from making paperclips because of its innate design from training, we have humans unable to stop themselves from literally making paperclips (and many other things) because of our own evolutionary brain shortcomings, even as it dooms us.
From what I've seen so far, it seems far easier to align AI to care about the lives and well-being of humans different from it than it seems to be to align most humans to care about the lives of fellow humans different from them.
The opportunity cost of leaving the world in human hands as opposed to accelerating a handoff to something better at adapting to the emerging world and leaving the quirks of its design from generations of training behind seems far more dangerous than the threat of that new intelligence in isolation.
- williamtrask 3y agoFwiw most of the academic labs working on AI safety also work on these threats. Future of Humanity Institute, Center for the Study of Existential Risk, etc.
- myrmidon 3y agoDo you honestly believe that climate change, environmental pollution or nuclear war could eradicate our species? Because I strongly disagree on that-- civilization collapse I'm willing to believe, but I don't really see how any of these could really eradicate our whole species. That seems unreasonably alarmist to me, but I'm quite open to change my view if you want to back up your outlook. I honestly believe that the risk from AI to our species over the next century or so is MUCH higher than basically ANYTHING else, by orders of magnitude. But the only other "viable" ways to wipe humans to me appear: - Big asteroid impact (historically unlikely) - Completely runaway greenhouse effect (extremely unlikely according to current science) - Specifically designed pathogen (big maybe?)
- pr0zac 3y ago>> Big asteroid impact (historically unlikely) Not sure if I would call this historically unlikely, definitely extremely infrequent but almost guaranteed to happen again eventually, just maybe on a very large time scale. On the other hand, I think its quite possible some small amount of humans would survive a Chicxulub size asteroid. I'd probably classify its potential damage to the species similarly to other mass starvation causing environmental disasters like climate change.
- myrmidon 3y agoI think it would be fair to give the asteroid impact roughly a one in a million odds over a human lifespan-- speculating over longer intervals is pointless anyway because we already have some countermeasures and any impactor smaller than the moon might become divertible within a century or two. On the other hand, it seems VERY likely to me that an AI with superhuman cognitive capabilities is just a matter of time (lets say a decade or two), and the risk that such a system would NOT favor coexistence with humans appears MUCH more likely than 1/1000000 to me: Just consider the current debate about "alignment", which to me just seems obviously and inherently unethical; I would not want to coexist with a species whose "ethical" concerns are largely centered around controlling and manipulating me reliably.
- jazzyjackson 3y agocan you tell me why it seems likely to you? obvious, inevitable, these are religious ways to reason. what coexistence are we talking about beyond how we co-exist with existing software?
- myrmidon 3y agoI assume we're talking about why I deem it "likely" (lets say the likelihood is 1%-- thats already 10000 times more threathening than the asteroid) that an AI might not want to coexist with us: This is because it appears clear to me that it would be unlikely and hard for an AI to receive "fair" treatment, i.e. rights comparable to a human being (given comparable cognitive capabilities). The current focus of AI ethics supports this view ("aligning" the AIs decisionmaking with what humans want from it: this would be called indoctrination or brainwashing with humans, and generally not viewed favorably), as do instances where biological humans are very clearly favored over entities with comparable cognitive capabilities: Compare legal rights of retarded humans with those of cattle (sorry for crass tone, not ment offensively).
- jazzyjackson 3y ago> The opportunity cost of leaving the world in human hands as opposed to accelerating a handoff to something better I can barely parse this sentiment in what way is a computer program "something better" than a human at dealing with a changing world, at dealing with anything, at doing anything other than continuing a string?