4 ms·

AI existential risk being the end of all or the majority of humanity? No, no I'm not, not as direct result of AGI evolving or seizing control like is widely spe

AI existential risk being the end of all or the majority of humanity? No, no I'm not, not as direct result of AGI evolving or seizing control like is widely speculated. The risk I see is always from humans being inept, greedy and stupid, resulting in deploying an AI system not fully understood where it shouldn't be, without a human in the loop. This Russia's dead man switch. All/most the existential risk theories predict AGI will evolve and reach super human intelligence, and have goals and drive, and some sort of motivation to do something, or just out of pure randomness of testing different envs, and stumbling on something that kills us all. However, were not even lose, and the systems are still contained to hardware, hardware that can be unplugged. AI is also as like to evolve the other way, to just it's simplest form, to survive as a few bits. There are too many hypothetical leaps and scenarios for this to worry me. Although they are interesting to read and fantasise about, and do foster some discussion about more immediate concerns about AI and even society. It's interesting to see how few users on hackernews are concerned about AI existential risk compared to similar questions on reddit, and youtube where everyone appeared to be afraid and those who disagreed that it is not a risk were downvoted.

arisAlexis 4y ago
Some comments. Motivation is called a reward function or goal. AI agents by default have a goal , you can read about the paperclip argument. About unplugging them th idea is that a very intelligent AI will convince us not ot unplug it through manipulation.
- winddude 4y ago
  > Motivation is called a reward function or goal. That's generally a singular goal in RL, and is human programmed. And also you would need to assume AGI won't have a programmed goal or motivation the same way as RL, and will pick it's own goals and need to discover it's own motivation. And the agents are still only interacting with envs programmed by humans. I find pretty much all of Bostrom's arguments outlandish and lacking a grounding in reality.

arisAlexis 4y ago

Some comments. Motivation is called a reward function or goal. AI agents by default have a goal , you can read about the paperclip argument. About unplugging them th idea is that a very intelligent AI will convince us not ot unplug it through manipulation.

winddude 4y ago
> Motivation is called a reward function or goal. That's generally a singular goal in RL, and is human programmed. And also you would need to assume AGI won't have a programmed goal or motivation the same way as RL, and will pick it's own goals and need to discover it's own motivation. And the agents are still only interacting with envs programmed by humans. I find pretty much all of Bostrom's arguments outlandish and lacking a grounding in reality.