8 ms·
However, to consider artificial intelligence to be a potential global catastrophe at all, let alone the single one requiring extra funding, is mostly unfounded.
by jlebar 8y ago
However, to consider artificial intelligence to be a potential global catastrophe at all, let alone the single one requiring extra funding, is mostly unfounded. We currently can barely even define what the actual risk is, let alone how to mitigate it.
Although I happen agree with you personally, I don't think we should commit the fallacy of assuming that because their position seems absurd to us that it comes from a place of bias or ignorance.
To the contrary, when I listen to Holden and other EA leaders talk, it's clear they've spent way more time thinking about this stuff than I have. He is and they are thoughtful and humble about exactly the questions you pose: How much should we weight "known good" good done today vs "potential good" done in the future, how confident should we be in our ability to predict the future, etc.
As one example, Open Phil has hired historians to conduct research on how well people have (in the past) been able to predict the future, precisely to inform their thinking in this regard.
Holden is also open about how his thoughts on the importance of AI alignment have changed over time.
Again, we can disagree with him (as I do). But we absolutely should not claim that because we disagree, his position must be out of a place of thoughtlessness or bias.
- jlebar 8y agoSee e.g. https://80000hours.org/podcast/episodes/holden-karnofsky-open-philanthropy/ https://80000hours.org/podcast/episodes/holden-karnofsky-ope...
- monktastic1 8y agoIt's perfectly reasonable for them to hold that view. The unfortunate thing is to insist that it's the most rational and/or correct view. To say that they're biased isn't an insult. We all have biases, and it's useful to recognize and admit them.
- mkirklions 8y ago>I don't think we should commit the fallacy of assuming that because their position seems absurd to us that it comes from a place of bias or ignorance. I had a really smart person talk about AI and how to deal with it. His conclusion was a gigantic let down. He was out of his element, hes an economist, but his conclusion was- Either AI is going to be peaceful, or its going to kill us and there is nothing we can do to stop it. Maybe most civilizations end like this, but why not look for third options?
- nordsieck 8y ago> Either AI is going to be peaceful, or its going to kill us and there is nothing we can do to stop it. Have you heard of the AI Box experiments? http://yudkowsky.net/singularity/aibox/ http://yudkowsky.net/singularity/aibox/ The problem of containing a hostile AI does not seem to be particularly tractable to me.
- mkirklions 8y agoThis could be really cool, but the conversation was impossible to read. The links/website is designed badly. Howd he let the AI out both times?
- nordsieck 8y ago> Howd he let the AI out both times? No one knows. That's the point. An AI will eventually be much smarter than a human, so it doesn't matter how the human succeeds - it's enough to know that a human has succeeded even once.
- mkirklions 8y agoHmm, would like a different human to try it. I read some of the dialog and the user being the gatekeeper talks openly about his inability to socialize that caused him to receive special social treatment until 10th grade. What if there were 2 gatekeepers, north korean style?
- 8y ago