4 ms·

How about someone actually articulate the details of how this supposed super intelligence will be built, what about it’s architecture means that it has no guard

How about someone actually articulate the details of how this supposed super intelligence will be built, what about it’s architecture means that it has no guardrails, how it will bypass numerous real world frictions to achieve its nefarious goals? Arguments that presuppose a god-like super intelligence are not useful. Sure, if we create a system that is all powerful I’d agree that it can destroy humanity. But specifically how are we going to build this?

ke88y 3y ago
>Arguments that presuppose a god-like super intelligence are not useful. Sure, if we create a system that is all powerful I’d agree that it can destroy humanity. But specifically how are we going to build this? Yeah, exactly. THIS is the type of x-risk talk that I find cringe-y and ego-centered. There are real risks. I've devoted my career to understanding and mitigating them, both big and small. There's also risk in risk mitigation, again, both big and small. Welcome to the world of actual engineering :) But a super-human god intelligence capable of destroying us all by virtue of that fact that it has an IQ over 9000? I have a book-length criticism of the entire premise and whether we should even waste breath talking about it before we even start the analysis of whether there is even the remotest scrap of evidence that were are anywhere near such a thing being possible. It's sci-fi. Which is fine. Just don't confuse compelling fiction with sound policy, science, or engineering.
ALittleLight 3y ago
If we knew how specifically this would be built, then it would already be built. That's really the only way we could know if something would work - if it has, in fact, worked. As is, there are only ideas and possibilities. The trajectory that AI is on signals risk. AI capabilities are increasing, approaching human level, and nothing suggests AI capabilities will stop there. If you agree that a super intelligence - i.e. something smarter than humanity, could destroy or dominate humanity (either by itself or in collusion with a small number of humans) and you agree that we don't currently have a plan to prevent this from happening, then it seems to me that you also agree with AI x-risk.

ke88y 3y ago

>Arguments that presuppose a god-like super intelligence are not useful. Sure, if we create a system that is all powerful I’d agree that it can destroy humanity. But specifically how are we going to build this? Yeah, exactly. THIS is the type of x-risk talk that I find cringe-y and ego-centered. There are real risks. I've devoted my career to understanding and mitigating them, both big and small. There's also risk in risk mitigation, again, both big and small. Welcome to the world of actual engineering :) But a super-human god intelligence capable of destroying us all by virtue of that fact that it has an IQ over 9000? I have a book-length criticism of the entire premise and whether we should even waste breath talking about it before we even start the analysis of whether there is even the remotest scrap of evidence that were are anywhere near such a thing being possible. It's sci-fi. Which is fine. Just don't confuse compelling fiction with sound policy, science, or engineering.

ALittleLight 3y ago

If we knew how specifically this would be built, then it would already be built. That's really the only way we could know if something would work - if it has, in fact, worked. As is, there are only ideas and possibilities. The trajectory that AI is on signals risk. AI capabilities are increasing, approaching human level, and nothing suggests AI capabilities will stop there. If you agree that a super intelligence - i.e. something smarter than humanity, could destroy or dominate humanity (either by itself or in collusion with a small number of humans) and you agree that we don't currently have a plan to prevent this from happening, then it seems to me that you also agree with AI x-risk.