6 ms·

What about corporations? They are self-sustaining non-human information processing systems with long term goals, and they are subject to selection pressure. The

by zardo 10y ago

What about corporations? They are self-sustaining non-human information processing systems with long term goals, and they are subject to selection pressure. They are rudimentary humans-in-the-loop artificial life.

TheOtherHobbes 10y ago
I think the most likely route to destructive AI is a corporation with an AI CEO - possibly one that takes over from a human CEO in a boardroom coup. Corporations already have legal personhood and act in their own interests. It's going to be much easier to automate and formalise business decision making than to develop a true general intelligence with a full spectrum of human characteristics. This may sound like science fiction, but as competence increases shareholders - who typically are only passingly interested in moral issues - are likely to demand the increased returns an AI CEO can bring.
- dragonwriter 10y ago
  > Corporations already have legal personhood and act in their own interests. But AIs don't, and corporate officers must be persons (natural persons, even -- a corporation can't be a corporate officer.)
  - cLeEOGPw 10y ago
    A person could be made a mediator of AI to overcome it.
- zardo 10y ago
  Many CEOs are already just trying to operate as a share price optimization algorithm. They don't choose to inject human values into their organizations.
  - oldmanjay 10y ago
    That you don't share their values does not make them non-human
    - js8 10y ago
      That they are human doesn't mean they cannot be a threat to humankind. ;-)
  - yourapostasy 10y ago
    > ...operate as a share price optimization algorithm... That in itself is not so bad, as extremely long-timeframe constraints (say, >50 years) upon such an algorithm could conceivably be consonant with current decision-making behavior that externalizes many input costs (employee overtime, environmental damage, etc.). Running the algorithm to pay out in very short timeframes (a month to a year) due to most CEOs' anticipated short tenure is what seems to cause undesirable optimizations.

by zardo 10y ago

TheOtherHobbes 10y ago

I think the most likely route to destructive AI is a corporation with an AI CEO - possibly one that takes over from a human CEO in a boardroom coup. Corporations already have legal personhood and act in their own interests. It's going to be much easier to automate and formalise business decision making than to develop a true general intelligence with a full spectrum of human characteristics. This may sound like science fiction, but as competence increases shareholders - who typically are only passingly interested in moral issues - are likely to demand the increased returns an AI CEO can bring.

dragonwriter 10y ago
> Corporations already have legal personhood and act in their own interests. But AIs don't, and corporate officers must be persons (natural persons, even -- a corporation can't be a corporate officer.)
- cLeEOGPw 10y ago
  A person could be made a mediator of AI to overcome it.
zardo 10y ago
Many CEOs are already just trying to operate as a share price optimization algorithm. They don't choose to inject human values into their organizations.
- oldmanjay 10y ago
  That you don't share their values does not make them non-human
  - js8 10y ago
    That they are human doesn't mean they cannot be a threat to humankind. ;-)
- yourapostasy 10y ago
  > ...operate as a share price optimization algorithm... That in itself is not so bad, as extremely long-timeframe constraints (say, >50 years) upon such an algorithm could conceivably be consonant with current decision-making behavior that externalizes many input costs (employee overtime, environmental damage, etc.). Running the algorithm to pay out in very short timeframes (a month to a year) due to most CEOs' anticipated short tenure is what seems to cause undesirable optimizations.