3 ms·

> Tesla [...] is clearly not a different class of intelligence from [...] corporations I believe he just used Tesla as an example of corporation not as somethi

> Tesla [...] is clearly not a different class of intelligence from [...] corporations I believe he just used Tesla as an example of corporation not as something new or special. > the point of the paperclip thought experiment is, broadly, that an AI's interests might not be aligned with our own and that misalignment may cause serious problems. But (some) corporations clearly do share this aspect to some extent. They maximize profit while environmental concerns are not prioritized. And he pointed out later that corporations are slightly more complex than just maximizing 'one' thing. But a range of things and most importantly profit.

ALittleLight 6y ago
Most, if not all, corporations are misaligned to humanity's well-being. That is, they want money, and that's not exactly the same thing as what would be best for humanity. This is okay though because corporations are the same class and kind of intelligence as the rest of humanity and so they can be (somewhat) predicted and constrained by our laws and norms. Because corporations are the same class of intelligence, they can't outsmart everyone. We can catch them when they do wrong and punish them and use laws and courts to control them. Because corporations are the same kind of intelligence they (usually) aren't going to do things we might consider insane or sociopathic. For example, if corporation X realized they might make a profit by taking out life insurance on workers and then getting them to do very risky things, X Corp probably wouldn't do it and if they did people would leak it and laws would be made to stop it. The paperclip thought experiment is an illustration of a different kind of intelligence that wouldn't be constrained by human morality or norms and would surpass human intelligence and not be constrained by laws or force either.
- plushpuffin 6y ago
  I think one of the big problems with trying to predict and constrain corporations is that they all basically have OCD. They are obsessed with whatever it is they do and they task all their agents with figuring out how to optimize and achieve that one thing (or multiple related things if they're organized into divisions). While the corporation might not possess superintelligence, trying to thwart it it when its actions are harmful to society is like having a crazy neighbor. You just want to relax when you're home from work; you can't dedicate all your free time and attention to constantly monitoring and pushing back against someone who is on a crusade to achieve a petty, limited goal and seemingly has no other interests to occupy their time.
  - ALittleLight 6y ago
    I think the extent to which corporations are or could be controlled is debatable. What I don't think is debatable though is that an AI with superhuman intelligence would be, necessarily, as controllable as corporations.

ALittleLight 6y ago

Most, if not all, corporations are misaligned to humanity's well-being. That is, they want money, and that's not exactly the same thing as what would be best for humanity. This is okay though because corporations are the same class and kind of intelligence as the rest of humanity and so they can be (somewhat) predicted and constrained by our laws and norms. Because corporations are the same class of intelligence, they can't outsmart everyone. We can catch them when they do wrong and punish them and use laws and courts to control them. Because corporations are the same kind of intelligence they (usually) aren't going to do things we might consider insane or sociopathic. For example, if corporation X realized they might make a profit by taking out life insurance on workers and then getting them to do very risky things, X Corp probably wouldn't do it and if they did people would leak it and laws would be made to stop it. The paperclip thought experiment is an illustration of a different kind of intelligence that wouldn't be constrained by human morality or norms and would surpass human intelligence and not be constrained by laws or force either.

plushpuffin 6y ago
I think one of the big problems with trying to predict and constrain corporations is that they all basically have OCD. They are obsessed with whatever it is they do and they task all their agents with figuring out how to optimize and achieve that one thing (or multiple related things if they're organized into divisions). While the corporation might not possess superintelligence, trying to thwart it it when its actions are harmful to society is like having a crazy neighbor. You just want to relax when you're home from work; you can't dedicate all your free time and attention to constantly monitoring and pushing back against someone who is on a crusade to achieve a petty, limited goal and seemingly has no other interests to occupy their time.
- ALittleLight 6y ago
  I think the extent to which corporations are or could be controlled is debatable. What I don't think is debatable though is that an AI with superhuman intelligence would be, necessarily, as controllable as corporations.