4 ms·

Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't

Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Similarly, you think that x-risk dismissal among the experts you agree with is not self-interested - but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested liars either. They would likely say, as you realize, "No, my dismissal of the concerns of other experts is legitimate!" The thing is, it doesn't require a lot of expertise to understand that there is actually an enormous risk and "experts" denying that are simply burying their head in the sand. AI, I use the term expansively, is making huge and rapid progress. We are approaching "human-level" intelligence and there is no guarantee, nor even any indication, that "human-level" is an upper limit. Systems as smart or smarter than we are, that are not controlled for the benefit of humanity, are an existential risk. There are two basic kinds of risk. First, an AI powered tyranny where a small number of humans dominate everything enabled by AI systems they control - every security camera watched by a tireless intelligence, drones piloted by perfectly loyal intelligences, etc. Second, that the AI systems are not effectively controlled and pursue goals or objectives that are incompatible with human existence. In either case the fact that the AI brings superhuman intelligence to bear means that humanity at large will be overmatched in terms of capabilities. It is completely possible that neither of these scenarios come to pass, but the fact that one of the scenarios might come to pass is what makes the situation an existential risk - humanity eternally subjected by an irreversible dictatorship, or simply destroyed. I'm confused about your first item that would convince you of AI risk, but I think the others should all be considered met by ChatGPT. Of course, ChatGPT isn't currently useful in developing new WMDs - but it is absolutely useful in helping to bring someone up to speed on new topics in many different domains. As capabilities advance, why wouldn't ChatGPT using GPT-5, or 6, or 10 be able to helpfully guide the creation of new and better WMDs? AI isn't currently causing the problems you are worried about but certainly seems on track to do so soon. "Risk" doesn't mean that we are currently in the process of being destroyed by AI, but it does mean that there is a non-negligible possibility that we will soon find ourselves in that process.

ke88y 3y ago
> Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Of course. I was merely explaining the reason why my response would focus on x-risk rather than better surveys, and why. > but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested liars either. They would likely say, as you realize, "No, my dismissal of the concerns of other experts is legitimate!" Okay. But I have more to gain than to lose by advocating for AI Safety, since that's what I work on. > The thing is, it doesn't require a lot of expertise to understand that there is actually an enormous risk My comment was about X-risk and AGI. Find any of a myriad of comments here where I agree there are real risks that should be taken seriously. Those risks -- the reasonable ones -- are not existential and have nothing to do with AGI. > Systems as smart or smarter than we are, that are not controlled for the benefit of humanity, are an existential risk... I am not an adherent to this religious dogma and have seen no empirical evidence that the powerful rationalisms people use to talk themselves into these positions has any basis in current or future reality. I want the LessWrong cult to have fewer followers in the halls of power precisely because I actually do give a damn about preventing the worst-case FEASIBLE outcomes, which have nothing to do with AGI bullshit. BTW: if we're all too busy worrying about AGI who's going prevent the pile of ridiculous should-be-illegal bullshit that's about to cause a bunch of real harm? No one. If you want conspiracies about intent, look at who's funding all this AGI x-risk bullshit. And you can be sure as hell of one thing: no one in big tech wants me giving Congress advice on what our AI and data privacy regulations should look like. They would MUCH prefer endless hearings on "risks" they might even know are bullshit nonsense. But why focus any effort on 100% risks of data privacy instrusions, copyright infringement, wanton anti-trust violations, and large-scale disinfo campaigns when there's a 1% risk of extinction, right? But anyways. I'm just some fool who has actually spent over a decade in the trenches trying to prevent specific bad outcomes. I'm sure the sci-fi essays and prognostications from famous CEOs are far more persuasive and entertaining and convicting than the ramblings of some nobody actually try to fix real problems with mostly boring solutions.
- ALittleLight 3y ago
  The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject it." I can't stop you from rejecting things out of hand, but you shouldn't be surprised that you come off as totally unpersuasive when you say and do these things. In the simplest form: 1. AI could become smarter than humanity. 2. AI is rapidly progressing towards super-human intelligence. 3. If AI is smarter than humanity it could destroy or dominate humanity. 4. We aren't certain AI won't destroy or dominate humanity. Therefore, there is some non-negligible existential risk from AI. Which of these points do you disagree with, or do you think the conclusion doesn't follow?

ke88y 3y ago

> Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Of course. I was merely explaining the reason why my response would focus on x-risk rather than better surveys, and why. > but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested liars either. They would likely say, as you realize, "No, my dismissal of the concerns of other experts is legitimate!" Okay. But I have more to gain than to lose by advocating for AI Safety, since that's what I work on. > The thing is, it doesn't require a lot of expertise to understand that there is actually an enormous risk My comment was about X-risk and AGI. Find any of a myriad of comments here where I agree there are real risks that should be taken seriously. Those risks -- the reasonable ones -- are not existential and have nothing to do with AGI. > Systems as smart or smarter than we are, that are not controlled for the benefit of humanity, are an existential risk... I am not an adherent to this religious dogma and have seen no empirical evidence that the powerful rationalisms people use to talk themselves into these positions has any basis in current or future reality. I want the LessWrong cult to have fewer followers in the halls of power precisely because I actually do give a damn about preventing the worst-case FEASIBLE outcomes, which have nothing to do with AGI bullshit. BTW: if we're all too busy worrying about AGI who's going prevent the pile of ridiculous should-be-illegal bullshit that's about to cause a bunch of real harm? No one. If you want conspiracies about intent, look at who's funding all this AGI x-risk bullshit. And you can be sure as hell of one thing: no one in big tech wants me giving Congress advice on what our AI and data privacy regulations should look like. They would MUCH prefer endless hearings on "risks" they might even know are bullshit nonsense. But why focus any effort on 100% risks of data privacy instrusions, copyright infringement, wanton anti-trust violations, and large-scale disinfo campaigns when there's a 1% risk of extinction, right? But anyways. I'm just some fool who has actually spent over a decade in the trenches trying to prevent specific bad outcomes. I'm sure the sci-fi essays and prognostications from famous CEOs are far more persuasive and entertaining and convicting than the ramblings of some nobody actually try to fix real problems with mostly boring solutions.

ALittleLight 3y ago
The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject it." I can't stop you from rejecting things out of hand, but you shouldn't be surprised that you come off as totally unpersuasive when you say and do these things. In the simplest form: 1. AI could become smarter than humanity. 2. AI is rapidly progressing towards super-human intelligence. 3. If AI is smarter than humanity it could destroy or dominate humanity. 4. We aren't certain AI won't destroy or dominate humanity. Therefore, there is some non-negligible existential risk from AI. Which of these points do you disagree with, or do you think the conclusion doesn't follow?