3 ms·

Ai isn't an alien. It doesn't have autonomy, will or desires. It's a set of numbers arranged in a particular way such that someone can use it to produce differe

by bko 4y ago

Ai isn't an alien. It doesn't have autonomy, will or desires. It's a set of numbers arranged in a particular way such that someone can use it to produce different numbers that can be interpreted as meaningful. That's all it is. How do you get from that to aliens controlling our climate? Give me one reasonable path. Will the numbers start changing themselves and pop out of the computer and float around and start moving things? This AGI doomsday nonsense is becoming more of a religion every day.

tester457 4y ago
> Give me one reasonable path. Actual AGI appears, and it has the ability to improve its source code. It becomes more intelligent than humanity and is misaligned to human goals. These goals will the mathematical representation of what the AI is trying to optimize, and there are many ways for this optimization to cause negative effects. We haven't even reached AGI, yet the LLM GPT4 was already able to act in the real world by manipulating a Task Rabbit Worker.
haswell 4y ago
> Ai isn't an alien. The goal of the alien analogy is to provide a thought experiment that allows us to expand how we think about AGI without getting stuck in preconceptions. It is not literally alien in the sense that most people think about when contemplating extraterrestrial life, but there are some strong parallels that are informative to consider. > It doesn't have autonomy, will or desires Current-generation AI tech is already teaching itself how to walk [0]. This is not a robot that follows deterministic rules pre-defined by a human, or a robot that was trained to walk. The robot trained itself. The AGI conversation is a conversation about the future where each of the specialized fields of AI converge. Much like we are massively complex interdependent systems, I suspect AGI will be the culmination of integrating focused fields of AI technology into a single system. Will and desire are human feelings that ultimately stem from our goal-oriented behavior. The absence of these feelings in a robot do not matter if the robot behaves in a goal-oriented way. Problems emerge when its goals are not aligned with our goals. This isn't about some hokey robot overlord scenario, but one where a robot sees humans much like humans see ants when going about our days. Most humans don't go out of their way to destroy ant colonies, but neither are we very concerned when we run the lawn mower over one. Malice is not required for a misalignment of goals to be a serious issue. > This AGI doomsday nonsense is becoming more of a religion every day. None of what I've described is inevitable, but all of it is possible. We're now in the moment where we'll decide what is next, and if we'll take this seriously enough to bake in appropriate safeguards. I think Stuart Russell has important things to say about this [1]. While I'm not a doomer, I'd strongly encourage you to dig deeper into this subject. I was shrugging most of this off for awhile, but things have changed. - [0] https://vcresearch.berkeley.edu/news/step-step-berkeley-robots-learn-walk-their-own-record-time https://vcresearch.berkeley.edu/news/step-step-berkeley-robo... - [1] https://www.youtube.com/watch?v=ISkAkiAkK7A https://www.youtube.com/watch?v=ISkAkiAkK7A

by bko 4y ago

tester457 4y ago

> Give me one reasonable path. Actual AGI appears, and it has the ability to improve its source code. It becomes more intelligent than humanity and is misaligned to human goals. These goals will the mathematical representation of what the AI is trying to optimize, and there are many ways for this optimization to cause negative effects. We haven't even reached AGI, yet the LLM GPT4 was already able to act in the real world by manipulating a Task Rabbit Worker.

haswell 4y ago

> Ai isn't an alien. The goal of the alien analogy is to provide a thought experiment that allows us to expand how we think about AGI without getting stuck in preconceptions. It is not literally alien in the sense that most people think about when contemplating extraterrestrial life, but there are some strong parallels that are informative to consider. > It doesn't have autonomy, will or desires Current-generation AI tech is already teaching itself how to walk [0]. This is not a robot that follows deterministic rules pre-defined by a human, or a robot that was trained to walk. The robot trained itself. The AGI conversation is a conversation about the future where each of the specialized fields of AI converge. Much like we are massively complex interdependent systems, I suspect AGI will be the culmination of integrating focused fields of AI technology into a single system. Will and desire are human feelings that ultimately stem from our goal-oriented behavior. The absence of these feelings in a robot do not matter if the robot behaves in a goal-oriented way. Problems emerge when its goals are not aligned with our goals. This isn't about some hokey robot overlord scenario, but one where a robot sees humans much like humans see ants when going about our days. Most humans don't go out of their way to destroy ant colonies, but neither are we very concerned when we run the lawn mower over one. Malice is not required for a misalignment of goals to be a serious issue. > This AGI doomsday nonsense is becoming more of a religion every day. None of what I've described is inevitable, but all of it is possible. We're now in the moment where we'll decide what is next, and if we'll take this seriously enough to bake in appropriate safeguards. I think Stuart Russell has important things to say about this [1]. While I'm not a doomer, I'd strongly encourage you to dig deeper into this subject. I was shrugging most of this off for awhile, but things have changed. - [0] https://vcresearch.berkeley.edu/news/step-step-berkeley-robots-learn-walk-their-own-record-time https://vcresearch.berkeley.edu/news/step-step-berkeley-robo... - [1] https://www.youtube.com/watch?v=ISkAkiAkK7A https://www.youtube.com/watch?v=ISkAkiAkK7A