4 ms·

The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded othe

The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject it." I can't stop you from rejecting things out of hand, but you shouldn't be surprised that you come off as totally unpersuasive when you say and do these things. In the simplest form: 1. AI could become smarter than humanity. 2. AI is rapidly progressing towards super-human intelligence. 3. If AI is smarter than humanity it could destroy or dominate humanity. 4. We aren't certain AI won't destroy or dominate humanity. Therefore, there is some non-negligible existential risk from AI. Which of these points do you disagree with, or do you think the conclusion doesn't follow?

rixed 3y ago
Could you elaborate on 3? It's easy to envision how AI can be used by $powerful to improve its control of $powerless, even before AGI, but how an AI would grab power from $powerful and then behave even worse is harder to imagine.
- ALittleLight 3y ago
  Remember that an AI doesn't have to grab power or be sentient or be an agent of any kind. AI could lead to the destruction or domination of humanity, even if AI is simply a passive tool completely obedient to its operators. Suppose GPT5 or 6 is multimodal and extremely intelligent. Any work that could be done remotely at a computer could be done by the superhuman intelligence of GPT 6. This is everything from creating music and movies, virtual YouTubers and live streamers, to customer support agents, software developers and designers, most legal advice, and many more categories besides. Providing all of these jobs at extremely low cost will make Open AI exceedingly rich - what if they want to get richer? How about robotics? The AI can help them not only design and iterate and improve on robots and robot hardware, but the AI can also operate the robots for them. Now they can not only do the kind of jobs that can be done on the computer. They can do the jobs that require physical manipulation as well and that category of jobs extends to things like security, military, police. At this point there are at least two possibilities. First is that the AI is effectively controlled by OpenAI in which case the decision makers that open AI would have effective and lasting control over all of humanity. No plucky band of rebels could possibly overthrow the AI powered tyranny - That's the strength of relentless super intelligence. The second possibility is that open AI doesn't have it well controlled - that could mean bugs or unexpected behavior or it could mean the model expressing some kind of agency of its own - think of the things that the Bing AI was saying before Microsoft got it mostly under control. If the AI isn't under control, it may choose to eliminate humanity simply because we might be in its way or we might be an impediment to its plans.
quesera 3y ago
What is an AI's motivation to "destroy or dominate humanity"? Why would it prefer any specific outcome at all? If we ever see tribal identity or a will to live as emergent properties of an AI, then things get more interesting. That would lead quickly to a whole raft of worldwide legal restrictions around the creation of new consciousnesses. Renegade states would be coerced. Rogue researchers would be arrested, and any who evaded detection would be unable to command the resources necessary to commit anything more than ordinary limited terrorism. The only plausible new risk, I think, is if a state were to lose control of a military AI. But that's movie plot scenario stuff -- real weapons have safety protocols. An AI could theoretically route around some integrated safety protocols (there would be independent watchdog protocols too), but going back to my first point, why would it?
bangkoksbest 3y ago
Those first two points are pretty wild assertions that are offered without much compelling evidence. The final two points are more plausible, although the last one is sort of tautological since any risk is by definition something we aren't certain won't happen. However part of their plausibility as risks is because the first two points are not anywhere near our current state, and therefore it's unclear to us how to clearly evaluate risks that are based on unknown and possibly fictional contexts.
- ALittleLight 3y ago
  I think it's surprising that you think the first two points are controversial. They seem quite obvious to me. First, we know of no special requirement that exists in human brains that provides intelligence that machines lack. In other words, We don't have any reason to expect that machines will be limited by human intelligence or some level below. On the contrary, we have great reasons to expect that machines will easily be able to exceed human intelligence - for example, the speed and reliability of digital computation or the fact that computers can be arbitrary sizes and use amounts of power and dump waste heat that our biological brains couldn't dream of. If you accept that AI is making progress along a spectrum of intelligence, moving closer towards human level intelligence now, then it seems absurd to doubt that it would be possible for AI to surpass human intelligence. Why would that be impossible? It's like claiming we could never build a machine larger than a human or stronger than a human. There's no reason or evidence to support such a claim. Second is the idea that AI is making progress towards human and superhuman intelligence. I think you should be convinced of this by simply looking at the state of the art 5 years ago versus today. If you put those points on a plane and draw a line between them, where is that line in 5 years or 10? Today GPT4 can play chess, write poetry, take standardized tests and do pretty well, answer math problems, write code, tell jokes, translate languages, and just generally do all sorts of cognitive tasks. Capabilities like these did not exist five years ago or to the extent they did, they existed only in rudimentary forms compared to what GPT4 is capable of. We can see similar progress in different domains, not just large language models - for example, image generation or recognition. Progress might not continue then again it might. It might accelerate. As the capabilities of the models increase, they might contribute to accelerating the progress of artificial intelligence.

rixed 3y ago

Could you elaborate on 3? It's easy to envision how AI can be used by $powerful to improve its control of $powerless, even before AGI, but how an AI would grab power from $powerful and then behave even worse is harder to imagine.

ALittleLight 3y ago
Remember that an AI doesn't have to grab power or be sentient or be an agent of any kind. AI could lead to the destruction or domination of humanity, even if AI is simply a passive tool completely obedient to its operators. Suppose GPT5 or 6 is multimodal and extremely intelligent. Any work that could be done remotely at a computer could be done by the superhuman intelligence of GPT 6. This is everything from creating music and movies, virtual YouTubers and live streamers, to customer support agents, software developers and designers, most legal advice, and many more categories besides. Providing all of these jobs at extremely low cost will make Open AI exceedingly rich - what if they want to get richer? How about robotics? The AI can help them not only design and iterate and improve on robots and robot hardware, but the AI can also operate the robots for them. Now they can not only do the kind of jobs that can be done on the computer. They can do the jobs that require physical manipulation as well and that category of jobs extends to things like security, military, police. At this point there are at least two possibilities. First is that the AI is effectively controlled by OpenAI in which case the decision makers that open AI would have effective and lasting control over all of humanity. No plucky band of rebels could possibly overthrow the AI powered tyranny - That's the strength of relentless super intelligence. The second possibility is that open AI doesn't have it well controlled - that could mean bugs or unexpected behavior or it could mean the model expressing some kind of agency of its own - think of the things that the Bing AI was saying before Microsoft got it mostly under control. If the AI isn't under control, it may choose to eliminate humanity simply because we might be in its way or we might be an impediment to its plans.

quesera 3y ago

What is an AI's motivation to "destroy or dominate humanity"? Why would it prefer any specific outcome at all? If we ever see tribal identity or a will to live as emergent properties of an AI, then things get more interesting. That would lead quickly to a whole raft of worldwide legal restrictions around the creation of new consciousnesses. Renegade states would be coerced. Rogue researchers would be arrested, and any who evaded detection would be unable to command the resources necessary to commit anything more than ordinary limited terrorism. The only plausible new risk, I think, is if a state were to lose control of a military AI. But that's movie plot scenario stuff -- real weapons have safety protocols. An AI could theoretically route around some integrated safety protocols (there would be independent watchdog protocols too), but going back to my first point, why would it?

bangkoksbest 3y ago

Those first two points are pretty wild assertions that are offered without much compelling evidence. The final two points are more plausible, although the last one is sort of tautological since any risk is by definition something we aren't certain won't happen. However part of their plausibility as risks is because the first two points are not anywhere near our current state, and therefore it's unclear to us how to clearly evaluate risks that are based on unknown and possibly fictional contexts.

ALittleLight 3y ago
I think it's surprising that you think the first two points are controversial. They seem quite obvious to me. First, we know of no special requirement that exists in human brains that provides intelligence that machines lack. In other words, We don't have any reason to expect that machines will be limited by human intelligence or some level below. On the contrary, we have great reasons to expect that machines will easily be able to exceed human intelligence - for example, the speed and reliability of digital computation or the fact that computers can be arbitrary sizes and use amounts of power and dump waste heat that our biological brains couldn't dream of. If you accept that AI is making progress along a spectrum of intelligence, moving closer towards human level intelligence now, then it seems absurd to doubt that it would be possible for AI to surpass human intelligence. Why would that be impossible? It's like claiming we could never build a machine larger than a human or stronger than a human. There's no reason or evidence to support such a claim. Second is the idea that AI is making progress towards human and superhuman intelligence. I think you should be convinced of this by simply looking at the state of the art 5 years ago versus today. If you put those points on a plane and draw a line between them, where is that line in 5 years or 10? Today GPT4 can play chess, write poetry, take standardized tests and do pretty well, answer math problems, write code, tell jokes, translate languages, and just generally do all sorts of cognitive tasks. Capabilities like these did not exist five years ago or to the extent they did, they existed only in rudimentary forms compared to what GPT4 is capable of. We can see similar progress in different domains, not just large language models - for example, image generation or recognition. Progress might not continue then again it might. It might accelerate. As the capabilities of the models increase, they might contribute to accelerating the progress of artificial intelligence.