3 ms·

> If people are more effective at doing their jobs with a tool, their productivity would increase meaning their wages would also increase. Does anyone think peo

> If people are more effective at doing their jobs with a tool, their productivity would increase meaning their wages would also increase. Does anyone think people in finance would be paid more if we got rid of excel? Excel made it possible to eliminate the manual parts of certain tasks that humans are bad at, and generally uninterested in doing. AGI removes the need for a human entirely. These considerations are not similar in substance or scope, and applying old models of understanding technology advancement is a dangerous game when exploring the potential realities of AGI. It’d be better to frame AGI conversations as if we had just discovered an alien species but had not yet made first contact. Assuming our understanding of biology on earth is sufficient to predict the outcomes of first contact would be similarly dangerous.

bko 4y ago
We alreat have AGI. Well not really but you can hire people. Do employees remove the need for management? Do you think you'll be able to prompt AGI "do business things" and it'll just carry on in it's virtual world that will somehow translate to real value?
- beernet 4y ago
  Per definition, AGI is not equivalent to human intelligence. Then again, intelligence itself is not well-defined either.
- haswell 4y ago
  The A in AGI stands for Artificial, and the result is by definition not human. Thinking about this problem in terms of human dynamics is a non-starter, and employee/manager relationships as we understand them are irrelevant or misleading at best. > Do you think you'll be able to prompt AGI "do business things" and it'll just carry on in its virtual world that will somehow translate to real value? Simply: yes. And much more. This is the general concern about AGI in a nutshell. If an alien ship shows up tomorrow, and its occupants express a desire to coexist/cooperate, and it becomes clear that they are significantly smarter than we are, what concerns come to mind? What if suddenly every company in existence is angling to benefit from this higher intelligence, and wants to proceed with full inter-species integration/collaboration full speed ahead, without fully understanding the goals of this other species? What if we later learned that this species prefers extremely hot climates and thrives on high CO2/low oxygen environments? Would we be so willing to participate in a short term business optimization initiative? What if we also learned that this species is remarkably similar to humans in that it will compete for various forms and degrees of power while ignoring the long term ramifications? Given what we know about our own species, the prospect that a super-intelligence might be built by and based on it should be terrifying.
  - bko 4y ago
    Ai isn't an alien. It doesn't have autonomy, will or desires. It's a set of numbers arranged in a particular way such that someone can use it to produce different numbers that can be interpreted as meaningful. That's all it is. How do you get from that to aliens controlling our climate? Give me one reasonable path. Will the numbers start changing themselves and pop out of the computer and float around and start moving things? This AGI doomsday nonsense is becoming more of a religion every day.
    - tester457 4y ago
      > Give me one reasonable path. Actual AGI appears, and it has the ability to improve its source code. It becomes more intelligent than humanity and is misaligned to human goals. These goals will the mathematical representation of what the AI is trying to optimize, and there are many ways for this optimization to cause negative effects. We haven't even reached AGI, yet the LLM GPT4 was already able to act in the real world by manipulating a Task Rabbit Worker.
    - haswell 4y ago
      > Ai isn't an alien. The goal of the alien analogy is to provide a thought experiment that allows us to expand how we think about AGI without getting stuck in preconceptions. It is not literally alien in the sense that most people think about when contemplating extraterrestrial life, but there are some strong parallels that are informative to consider. > It doesn't have autonomy, will or desires Current-generation AI tech is already teaching itself how to walk [0]. This is not a robot that follows deterministic rules pre-defined by a human, or a robot that was trained to walk. The robot trained itself. The AGI conversation is a conversation about the future where each of the specialized fields of AI converge. Much like we are massively complex interdependent systems, I suspect AGI will be the culmination of integrating focused fields of AI technology into a single system. Will and desire are human feelings that ultimately stem from our goal-oriented behavior. The absence of these feelings in a robot do not matter if the robot behaves in a goal-oriented way. Problems emerge when its goals are not aligned with our goals. This isn't about some hokey robot overlord scenario, but one where a robot sees humans much like humans see ants when going about our days. Most humans don't go out of their way to destroy ant colonies, but neither are we very concerned when we run the lawn mower over one. Malice is not required for a misalignment of goals to be a serious issue. > This AGI doomsday nonsense is becoming more of a religion every day. None of what I've described is inevitable, but all of it is possible. We're now in the moment where we'll decide what is next, and if we'll take this seriously enough to bake in appropriate safeguards. I think Stuart Russell has important things to say about this [1]. While I'm not a doomer, I'd strongly encourage you to dig deeper into this subject. I was shrugging most of this off for awhile, but things have changed. - [0] https://vcresearch.berkeley.edu/news/step-step-berkeley-robots-learn-walk-their-own-record-time https://vcresearch.berkeley.edu/news/step-step-berkeley-robo... - [1] https://www.youtube.com/watch?v=ISkAkiAkK7A https://www.youtube.com/watch?v=ISkAkiAkK7A

bko 4y ago

We alreat have AGI. Well not really but you can hire people. Do employees remove the need for management? Do you think you'll be able to prompt AGI "do business things" and it'll just carry on in it's virtual world that will somehow translate to real value?

beernet 4y ago
Per definition, AGI is not equivalent to human intelligence. Then again, intelligence itself is not well-defined either.
haswell 4y ago
The A in AGI stands for Artificial, and the result is by definition not human. Thinking about this problem in terms of human dynamics is a non-starter, and employee/manager relationships as we understand them are irrelevant or misleading at best. > Do you think you'll be able to prompt AGI "do business things" and it'll just carry on in its virtual world that will somehow translate to real value? Simply: yes. And much more. This is the general concern about AGI in a nutshell. If an alien ship shows up tomorrow, and its occupants express a desire to coexist/cooperate, and it becomes clear that they are significantly smarter than we are, what concerns come to mind? What if suddenly every company in existence is angling to benefit from this higher intelligence, and wants to proceed with full inter-species integration/collaboration full speed ahead, without fully understanding the goals of this other species? What if we later learned that this species prefers extremely hot climates and thrives on high CO2/low oxygen environments? Would we be so willing to participate in a short term business optimization initiative? What if we also learned that this species is remarkably similar to humans in that it will compete for various forms and degrees of power while ignoring the long term ramifications? Given what we know about our own species, the prospect that a super-intelligence might be built by and based on it should be terrifying.
- bko 4y ago
  Ai isn't an alien. It doesn't have autonomy, will or desires. It's a set of numbers arranged in a particular way such that someone can use it to produce different numbers that can be interpreted as meaningful. That's all it is. How do you get from that to aliens controlling our climate? Give me one reasonable path. Will the numbers start changing themselves and pop out of the computer and float around and start moving things? This AGI doomsday nonsense is becoming more of a religion every day.
  - tester457 4y ago
    > Give me one reasonable path. Actual AGI appears, and it has the ability to improve its source code. It becomes more intelligent than humanity and is misaligned to human goals. These goals will the mathematical representation of what the AI is trying to optimize, and there are many ways for this optimization to cause negative effects. We haven't even reached AGI, yet the LLM GPT4 was already able to act in the real world by manipulating a Task Rabbit Worker.
  - haswell 4y ago
    > Ai isn't an alien. The goal of the alien analogy is to provide a thought experiment that allows us to expand how we think about AGI without getting stuck in preconceptions. It is not literally alien in the sense that most people think about when contemplating extraterrestrial life, but there are some strong parallels that are informative to consider. > It doesn't have autonomy, will or desires Current-generation AI tech is already teaching itself how to walk [0]. This is not a robot that follows deterministic rules pre-defined by a human, or a robot that was trained to walk. The robot trained itself. The AGI conversation is a conversation about the future where each of the specialized fields of AI converge. Much like we are massively complex interdependent systems, I suspect AGI will be the culmination of integrating focused fields of AI technology into a single system. Will and desire are human feelings that ultimately stem from our goal-oriented behavior. The absence of these feelings in a robot do not matter if the robot behaves in a goal-oriented way. Problems emerge when its goals are not aligned with our goals. This isn't about some hokey robot overlord scenario, but one where a robot sees humans much like humans see ants when going about our days. Most humans don't go out of their way to destroy ant colonies, but neither are we very concerned when we run the lawn mower over one. Malice is not required for a misalignment of goals to be a serious issue. > This AGI doomsday nonsense is becoming more of a religion every day. None of what I've described is inevitable, but all of it is possible. We're now in the moment where we'll decide what is next, and if we'll take this seriously enough to bake in appropriate safeguards. I think Stuart Russell has important things to say about this [1]. While I'm not a doomer, I'd strongly encourage you to dig deeper into this subject. I was shrugging most of this off for awhile, but things have changed. - [0] https://vcresearch.berkeley.edu/news/step-step-berkeley-robots-learn-walk-their-own-record-time https://vcresearch.berkeley.edu/news/step-step-berkeley-robo... - [1] https://www.youtube.com/watch?v=ISkAkiAkK7A https://www.youtube.com/watch?v=ISkAkiAkK7A